Azure Databricks

Role & responsibilities

Job Title:Azure Databricks Engineer / Senior Azure Databricks Developer

Location:Pune

Experience:6-10 Years

Preferred candidate profile

Job Summary:

We are looking for a skilled Azure Databricks Engineer with strong expertise in building scalable data pipelines, data transformation, and cloud-based analytics solutions on the Azure platform. The candidate will work closely with data engineers, architects, and business stakeholders to design, develop, and optimize modern data solutions using Azure Databricks, PySpark, and Azure Data Services.

Key Responsibilities:

Design, develop, and maintain ETL/ELT pipelines using Azure Databricks and PySpark.

Build scalable data processing solutions for batch and real-time workloads.

Integrate data from multiple sources including databases, APIs, and cloud storage.

Develop and optimize Spark jobs for performance and scalability.

Implement data quality, validation, and monitoring frameworks.

Work with Azure services to build end-to-end data engineering solutions.

Collaborate with business teams to understand data requirements and deliver analytics-ready datasets.

Support CI/CD implementation and deployment automation.

Troubleshoot production issues and optimize existing data pipelines.

Required Skills:

5+ years of Data Engineering experience.

Strong hands-on experience with Azure Databricks.

Expertise in PySpark and Apache Spark.

Strong SQL programming skills.

Experience with:

Azure Data Factory (ADF)

Azure Data Lake Storage (ADLS Gen2)

Azure Synapse Analytics

Azure Blob Storage

Azure Key Vault

Azure DevOps

Experience in data modeling and ETL development.

Knowledge of Delta Lake architecture.

Understanding of performance tuning and optimization techniques.

Experience with Git version control.

Preferred Skills:

Azure Data Engineer Certification (DP-203).

Experience with Kafka/Event Hub.

Knowledge of Scala or Python programming.

Experience in Data Warehousing concepts.

Exposure to Snowflake is a plus.

Experience with Agile/Scrum methodologies.