Technical Lead-Data Engg

Area(s) of responsibility Key Responsibilities

• Design and develop scalable ETL/ELT pipelines using PySpark, Spark SQL, Python, and Databricks.
• Develop complex transformation logic across Bronze, Silver, and Gold layers.
• Implement reusable transformation frameworks and common data processing utilities.
• Optimize transformation workloads for performance, scalability, and cost.
• Apply data quality, validation, and business-rule checks throughout the pipeline.
• Enterprise Data Modelling
• Design scalable data models using:
• Dimensional modelling
• Star/Snowflake schemas
• Data Vault 2.0
• Canonical data models
• Develop curated data products and analytical datasets using Databricks SQL and Delta Lake.
• Support enterprise-wide data harmonization across clinical, regulatory, safety, and commercial domains.
• Experience supporting data platforms across:
• Clinical trials – EDC, CDMS, CTMS
• Regulatory and submission systems
• Pharmacovigilance and safety
• Commercial analytics
• Real-World Evidence (RWE)
• Ensure adherence to relevant regulatory and data governance requirements including GxP, 21 CFR Part 11, HIPAA/GDPR, and ALCOA+ principles.

Required Qualifications

• 10–15 years of experience in Data Engineering and enterprise data platforms.
• Strong hands-on experience with Databricks, Apache Spark, PySpark, Python, and SQL.
• 4+ years of experience with Databricks/Spark-based data engineering preferred.
• Strong experience in:
• Data migration and modernization
• ETL/ELT development
• Batch and streaming data pipelines
• Delta Lake
• Databricks Workflows
• Unity Catalog
• Data modelling
• Data quality and reconciliation
• Experience with cloud platforms – AWS/Azure/GCP.
• Strong understanding of CI/CD, Git, Jenkins, and DevOps practices.
• Experience working in Life Sciences/Pharma environments preferred.