Azure Data Bricks Engineer_6+yrs

About The Opportunity A fast-scaling technology consultancy in the Enterprise Data & AI Engineering space, we help global clients modernize data architectures, operationalize machine learning, and drive intelligent automation using Azure cloud-native tools. You’ll work on high-impact data engineering projects leveraging Azure Databricks to build scalable, secure, and governed data pipelines for Fortune 500 and mid-market clients across BFSI, Retail, and Healthcare.

Role & Responsibilities

• Design and deploy production-grade data pipelines using Azure Databricks, PySpark, and Delta Lake for batch and streaming workloads.
• Optimize Databricks clusters for cost, performance, and concurrency; implement CI/CD workflows via Azure DevOps or GitHub Actions.
• Integrate Databricks with Azure Data Factory, Synapse, ADLS Gen2, and Power BI to enable end-to-end analytics and ML lifecycle.
• Enforce data governance and lineage using Unity Catalog, manage secrets via Key Vault, and implement RBAC for secure multi-tenant workspaces.
• Collaborate with Data Scientists to productionize ML models—feature engineering, training, and batch scoring pipelines.
• Document architecture patterns, performance benchmarking, and operational runbooks for client handover and internal knowledge sharing.

Skills & Qualifications Must-Have

• Azure Databricks
• PySpark
• Delta Lake
• Azure Data Factory
• ADLS Gen2
• Unity Catalog
• CI/CD (Azure DevOps / GitHub Actions)
• SQL

Preferred

• MLflow
• Feature Store (Azure ML or Feast)
• Knowledge of ML model deployment patterns

Benefits & Culture Highlights

• Work with Fortune 500 clients across global industries and solve real-world data challenges.
• On-site role with collaborative, high-ownership culture—no micromanagement, fast execution.
• Access to certified Azure training, cloud credits, and mentorship from senior engineers. Skills: databricks,ci,cd,python,azure synapse,sql,azure