Azure Data Engineer

Senior Azure Data Engineer (8+ Years Experience)

Location: Remote

Salary - Up to 15 LPA

Job Summary:

We are seeking a highly skilled Senior Azure Data Engineer with 8+ years of experience in designing, implementing, and optimizing data solutions on Azure Cloud. The ideal candidate will have strong expertise in Azure Data Factory (ADF), Databricks, and Unity Catalog, along with a deep understanding of data engineering best practices, ETL pipelines, and data governance.

Key Responsibilities:

Design, develop, and maintain scalable data pipelines and ETL processes using Azure Data Factory (ADF) and Databricks.

Implement Unity Catalog for centralized data governance, access control, and metadata management across Databricks.

Optimize data ingestion, transformation, and processing for structured and unstructured data sources.

Collaborate with data scientists, analysts, and business teams to ensure high-quality data availability and accessibility.

Implement data security, compliance, and best practices following enterprise guidelines and industry standards.

Monitor and troubleshoot performance, reliability, and data quality issues in Azure-based data solutions.

Develop and maintain CI/CD pipelines for data workflows using DevOps and Infrastructure-as-Code (IaC) principles.

Work with stakeholders to define data strategy, architecture, and best practices for efficient data operations.

Required Skills Qualifications:

8+ years of experience in data engineering with a strong focus on Azure Cloud.

Expertise in Azure Data Factory (ADF) for building and orchestrating data pipelines.

Hands-on experience with Databricks (PySpark, SQL, Delta Lake) for data processing and analytics.

Deep understanding of Unity Catalog for data governance, access control, and cataloging.

Strong experience in SQL, Python, and Spark for data transformation and analytics.

Knowledge of Azure Synapse Analytics, Azure Data Lake, and other Azure data services.

Experience with CI/CD, DevOps, Infrastructure-as-Code (Terraform, ARM templates).

Familiarity with data security, compliance (GDPR, HIPAA, etc.), and governance best practices.

Strong problem-solving skills with the ability to work in a fast-paced, agile environment.

Excellent communication and collaboration skills to work effectively with cross-functional teams.

Preferred Qualifications:

Experience in building data lakes, data mesh, or modern data platforms.

Knowledge of Machine Learning, AI, or real-time data streaming (Kafka, Event Hubs).

Certifications such as Microsoft Certified: Azure Data Engineer Associate (DP-203) or Databricks certifications.

A collaborative and innovative work environment.

Competitive salary, benefits, and professional growth opportunities.