Lead Data Engineer | Databricks

Work Location: Bengaluru

Mode: WFO

Exp: 8+ Yrs

Note: need someone who can talk to business stakeholders with experience of Product Owner, as this person will lead a team of developers and business analysts and should design/plan/execute/control the execution

Data Hub Lead - JD

Professional Work Experience

• 8+ years of experience in Data Engineering, Data Platforms, Analytics, or Cloud Data Architecture, with 3–5+ years of hands-on Databricks experience.
• Strong expertise in Databricks, Apache Spark, PySpark, Delta Lake, Unity Catalog, Databricks Workflows, and SQL.
• Proven experience designing and implementing modern Lakehouse architectures, large-scale data pipelines, and real-time data processing solutions.
• Deep knowledge of cloud platforms such as Azure (preferred), including data services, storage, networking, and security.
• Hands-on experience with ETL/ELT frameworks, data modeling, DataOps, CI/CD pipelines, and infrastructure automation.
• Experience building data products that support BI, Advanced Analytics, Machine Learning, and Generative AI use cases.
• Strong understanding of data governance, metadata management, data quality, lineage, privacy, and compliance frameworks.
• Proven track record leading technical teams, mentoring engineers, and delivering large-scale enterprise data transformation programs.
• Experience with DevOps/DataOps tools such as Azure DevOps, GitHub, Terraform, Jenkins, and Kubernetes is highly desirable.
• Databricks certifications such as Databricks Certified Data Engineer Professional, Databricks Certified Professional Data Architect, or equivalent are preferred.

Strategic Purpose of the Position

Due to expansion and increased complexity of Datahub solutions in the organization, Lead role requires to lead various regional business initiatives and ensuring data accuracy and security within datahub landscape.

Position Description in Brief and Approx Spend of Time in %

Lead team of 1-2 external developers, provide guidance in terms of project and tasks prioritization - 20%

Work with various Global functions stakeholders to gather their data needs and consulting them on best-in-class solutions - 20%

Manage delivery of the projects - 20%

Create development tickets, manage backlog of open tickets and perform - 20%

Understanding technology stack and help in designing of solutions in Cloud and On-prem - 20%