Azure Data Engineer

Role & responsibilities :

Databricks (Spark SQL, Delta Lake, Unity Catalog)

SQL & LargeScale Data Processing Performance

Data Pipeline Design & Lakehouse Architecture

Data Modeling (Fact/Dimension, Analytics Readiness)

Git, CI/CD, Governance & Security

dbt (Models, Incremental Logic Working Knowledge)

• Design, develop, and maintain largescale batch data pipelines using Databricks and Delta Lake.
• Build and optimize Spark SQLbased transformations for highvolume, highperformance data processing.
• Implement Delta Lake best practices, including partitioning, OPTIMIZE, ZORDER, VACUUM, and timetravel.
• Design and manage curated fact and dimension tables aligned with Lakehouse architecture principles.
• Apply data modeling techniques to support analytical, BI, and reporting use cases.
• Ensure data reliability, consistency, and correctness across endtoend pipelines.
• Implement data governance, access control, and security policies using Unity Catalog.
• Build and manage CI/CD pipelines for Databricks notebooks, jobs, and SQL artifacts using Gitbased workflows.
• Monitor, troubleshoot, and optimize production pipelines for performance, cost, and scalability.
• Collaborate with analytics teams and support dbt integrations where required (good to have).

Preferred candidate profile

• Own endtoend data engineering pipelines from raw ingestion to curated datasets.
• Ensure productiongrade reliability, monitoring, and operational support of Databricks workloads.
• Participate in architecture design and platform decisionmaking.
• Optimize Spark and SQL workloads to meet SLA and cost targets.
• Mentor junior engineers and promote data engineering best practices.
• Work closely with analytics engineers and business stakeholders to deliver trusted data products.