MODULE LEAD - Azure Databricks

Senior Azure Data Engineer

Experience

8+ Years

Location

Bangalore / Hybrid / Client Location

Job Summary

We are looking for an experienced Senior Azure Data Engineer to design, develop, and support enterprise-scale data platforms on Microsoft Azure. The ideal candidate should have strong expertise in Azure data technologies, data engineering frameworks, and cloud-based analytics solutions. The role involves working closely with business stakeholders, architects, and engineering teams to deliver scalable and high-performance data solutions

Required Skills

Azure Databricks

PySpark / Apache Spark

Python

SQL / Spark SQL

Azure Data Factory (ADF)

Azure Data Lake Storage (ADLS Gen2)

Azure Synapse Analytics

Delta Lake

Unity Catalog

Azure SQL Database

Azure Blob Storage

Azure DevOps

GitHub & CI/CD Pipelines

Data Governance & Data Quality Frameworks

Key Responsibilities

Design, develop, and maintain scalable data pipelines using Azure Databricks, PySpark, Python, and SQL.

Gather and analyse business requirements and translate them into technical solutions.

Build and optimize ETL/ELT processes and data integration frameworks.

Implement data governance, validation, monitoring, and security standards.

Develop and manage CI/CD pipelines for automated deployments.

Perform code reviews, production support, troubleshooting, and root cause analysis.

Collaborate with business, architecture, and support teams to ensure successful project delivery.

Prepare and maintain technical documentation, architecture diagrams, and design documents.

Good to Have

Azure Logic Apps

ServiceNow Integration

BigPanda Integration

Remedyforce Integration

HDInsight / Hive

SSIS / SSRS

Celonis OCPM

Databricks Genie

GenAI Solutions & GitHub Copilot Experience

Education

Bachelors or Masters degree in Computer Science, Information Technology, Engineering, or related field.

Preferred Candidate Profile

Strong experience in Azure Data Engineering and Analytics.

Experience leading technical design and implementation activities.

Excellent problem-solving and stakeholder management skills.

Hands-on experience with cloud-based data platforms and enterprise-scale solutions.

Strong communication and documentation skills.