Senior Databricks Data Engineer ( Pharma / Life Science experience )

Job Title:

Senior Databricks Data Engineer (1 Resource)

Experience: 6+ Years

Location: Remote

Working hours: 5:30 pm - 2:30 am IST

Job Description We are looking for a Senior Databricks Data Engineer with strong hands-on experience in building scalable data pipelines and Lakehouse solutions using Databricks, PySpark, Spark SQL, Delta Lake, and cloud platforms. The role requires solid engineering skills, performance tuning expertise, and the ability to work closely with technical and business stakeholders. Experience in pharma or life sciences domains is mandatory.

Key Responsibilities 1. Design, develop, and maintain scalable data pipelines using Databricks, PySpark, Spark SQL, and Delta Lake. 2. Implement Lakehouse and Medallion architecture patterns for reliable, analytics-ready data products. 3. Build ETL/ELT pipelines integrating data from multiple sources such as APIs, databases, files, and streaming platforms. 4. Optimize Spark jobs, Delta tables, clusters, and workflows for performance, reliability, and cost efficiency. 5. Implement data quality, governance, access control, and lineage practices using Unity Catalog or equivalent tools. 6. Collaborate with architects, analysts, data scientists, and business teams; mentor junior engineers and promote engineering best practices.

Required Skills 1. 6+ years of data engineering experience, including strong hands-on experience with Databricks. 2. Strong proficiency in PySpark, Spark SQL, Python, SQL, and distributed data processing concepts. 3. Experience with Delta Lake, Databricks Workflows, Delta Live Tables, Unity Catalog, and Databricks SQL. 4. Good understanding of Lakehouse architecture, data warehousing, data modelling, and ETL/ELT design patterns. 5. Experience with cloud platforms such as Azure or AWS and orchestration tools such as ADF, Airflow, or dbt. 6. Familiarity with Git, CI/CD, testing, release management, and production support practices. 7. Strong communication, problem-solving, stakeholder management, and mentoring skills.

Added Advantage

• Experience in pharma, life sciences, healthcare, or regulated environments such as GxP, GDPR, or HIPAA.
• Databricks Certified Data Engineer Associate or Professional certification.
• Experience supporting analytics, AI/ML, or data product use cases on the Databricks platform.