Databrick Data Engineer

Role & responsibilities

• Design, build and maintain scalable ETL/ELT pipelines using Databricks, PySpark, Spark SQL and Delta Lake.
• Implement Lakehouse solutions using Bronze/Silver/Gold architecture.
• Optimise Spark jobs, clusters and workloads for performance and cost.
• Build Databricks Workflows/Jobs and integrate with Airflow, ADF or equivalent orchestration tools.
• Implement data-quality checks, monitoring and observability.
• Support ML feature pipelines and MLflow-based workflows.
• Work with Unity Catalog for governance, access control and lineage.
• Develop robust data models with business and analytics teams.
• Support Git-based CI/CD and Databricks Repos.
• Troubleshoot production data-pipeline issues and perform root-cause analysis.

Preferred candidate profile

• 5 to 8 years of Data Engineering experience
• Minimum 2 to 3 years of hands-on Databricks experience
• PySpark
• Spark SQL
• Python
• Delta Lake
• Lakehouse / Medallion Architecture
• Azure Databricks / AWS / GCP
• Unity Catalog
• Databricks Workflows / Jobs
• Spark performance and cluster optimisation
• Strong SQL and data modelling
• Airflow / ADF / dbt
• Git and CI/CD
• Data governance, security and lineage