Azure Data Engineer

Hiring for One of the topmost MNC!!

ROLE: Azure Data Engineer

Exp: 5 to 8 YRS

Location: PAN INDIA

Work mode: 5 Days work from office

Notice Period: 0-30 Days Only

Job Overview:

We are looking for an experienced Azure Data Engineer with strong expertise in Azure Data Factory (ADF), Azure Databricks, PySpark, Python, and Unity Catalog. The candidate will be responsible for designing, developing, and maintaining scalable data pipelines and data processing solutions on the Microsoft Azure platform.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory and Azure Databricks.
• Develop complex data transformations and processing logic using PySpark.
• Write clean, efficient, and reusable Python code for data engineering workloads.
• Build and optimize Databricks notebooks, jobs, workflows, and data pipelines.
• Implement data ingestion pipelines from various sources including databases, APIs, files, and cloud storage.
• Work with Azure Data Lake Storage Gen2 (ADLS) for data storage and processing.
• Implement Delta Lake tables and develop reliable lakehouse data solutions.
• Use Unity Catalog for centralized data governance, access control, cataloging, and lineage.
• Manage catalogs, schemas, tables, views, external locations, and permissions using Unity Catalog.
• Implement secure data access using appropriate Azure and Databricks security mechanisms.
• Develop incremental and full-load data pipelines using appropriate ingestion strategies.
• Implement error handling, logging, monitoring, and alerting for production pipelines.
• Optimize PySpark jobs and Databricks workloads for performance and cost.
• Troubleshoot pipeline failures, data quality issues, and production incidents.

--
Thanks & Regards
Nikhil C
IT Staffing Analyst
Black and White Business Solutions Pvt.Ltd.
Bangalore,Karnataka,India.
[email removed]|www.blackwhite.in