Technical Lead-Data Engg

Area(s) of responsibility

Key Responsibilities
•
-    Design and develop scalable ETL/ELT pipelines using PySpark, Spark SQL, Python, and Databricks.
-    Develop complex transformation logic across Bronze, Silver, and Gold layers.
-    Implement reusable transformation frameworks and common data processing utilities.
-    Optimize transformation workloads for performance, scalability, and cost.
-    Apply data quality, validation, and business-rule checks throughout the pipeline.
•    Enterprise Data Modelling
-    Design scalable data models using:
-    Dimensional modelling
-    Star/Snowflake schemas
-    Data Vault 2.0
-    Canonical data models
-    Develop curated data products and analytical datasets using Databricks SQL and Delta Lake.
-    Support enterprise-wide data harmonization across clinical, regulatory, safety, and commercial domains.
•    Experience supporting data platforms across:
-    Clinical trials – EDC, CDMS, CTMS
-    Regulatory and submission systems
-    Pharmacovigilance and safety
-    Commercial analytics
-    Real-World Evidence (RWE)
•    Ensure adherence to relevant regulatory and data governance requirements including GxP, 21 CFR Part 11, HIPAA/GDPR, and ALCOA+ principles.
Required Qualifications
•    10–15 years of experience in Data Engineering and enterprise data platforms.
•    Strong hands-on experience with Databricks, Apache Spark, PySpark, Python, and SQL.
•    4+ years of experience with Databricks/Spark-based data engineering preferred.
•    Strong experience in:
•    Data migration and modernization
•    ETL/ELT development
•    Batch and streaming data pipelines
•    Delta Lake
•    Databricks Workflows
•    Unity Catalog
•    Data modelling
•    Data quality and reconciliation
•    Experience with cloud platforms – AWS/Azure/GCP.
•    Strong understanding of CI/CD, Git, Jenkins, and DevOps practices.
•    Experience working in Life Sciences/Pharma environments preferred.