Azure Databricks Engineering Lead

We are seeking an experienced Azure Databricks Engineering Lead to provide technical leadership and overall engineering ownership for an enterprise Azure Databricks data platform. This role will lead the design, development, optimization, and evolution of the Databricks ecosystem, while establishing engineering standards, reusable patterns, and best practices across the team.

The ideal candidate will bring 10–15 years of data engineering and data platform experience, with deep hands-on expertise in Azure Databricks, Apache Spark, Scala, SQL, ADLS, and batch/streaming architectures. This individual will serve as the senior technical authority for the Databricks team, driving technical decisions, mentoring engineers, guiding architecture, and ensuring delivery of scalable, secure, reliable, and high-performing data solutions.

Technical — Must Have
Azure Databricks — advanced hands-on development and platform expertise
Azure Data Lake Storage (ADLS)
Apache Spark — architecture, core concepts, optimization, and performance tuning
Scala
SQL — advanced development and performance tuning
Data Streaming — Spark Structured Streaming and enterprise streaming patterns
Enterprise-scale data engineering and data platform architecture
Experience establishing reusable frameworks, engineering standards, and development patterns


Required Experience

• 10–15 years of progressive experience in Data Engineering, Data Warehousing, and/or enterprise data platforms.
• 5+ years of hands-on Databricks and/or Spark experience, preferably within Azure.
• Proven experience designing and delivering enterprise-grade ETL/ELT and data processing solutions at scale.
• Deep experience with Azure Databricks, Spark, Scala, SQL, Azure Data Lake, and batch/streaming architectures.
• Demonstrated experience serving as a technical or engineering lead for a data engineering team.
• Experience defining architecture, engineering standards, reusable frameworks, and development patterns.
• Proven ability to lead complex initiatives from strategy and design through implementation and production support.
• Strong communication skills and experience collaborating with architects, product owners, analysts, QA, and upstream/downstream teams.