Senior Data Engineer (Azure)

We are seeking a highly skilled Senior Data Engineer with strong experience in Azure, Databricks, and large-scale data processing to design, build, and optimize enterprise-grade data solutions. The role requires a hands-on professional with deep technical expertise in data architecture, distributed processing, and cloud-native data platforms, along with exposure to AI/ML data use cases and modern Lakehouse practices. The ideal candidate will play a key role in developing scalable, secure, and reliable data pipelines and platforms that enable analytics, reporting, and AI-driven business capabilities.

Responsibilities:

• Design, develop, and maintain scalable data architecture and data engineering solutions on Azure and Databricks.
• Build robust batch and streaming data pipelines using Apache Spark, PySpark, Delta Lake, SQL, and related technologies.
• Develop and optimize complex data processing frameworks for structured, semi-structured, and high-volume datasets.
• Implement enterprise data lakehouse solutions aligned to business, analytics, and AI/ML requirements.
• Ensure data pipelines are reliable, performant, reusable, and built with strong engineering standards.
• Collaborate with business teams, data architects, analysts, data scientists, and cloud engineers to translate requirements into technical solutions.
• Support AI and ML initiatives by creating curated, governed, and feature-ready datasets.
• Work on performance tuning, troubleshooting, and optimization of Databricks workloads and Spark jobs.
• Establish and maintain data quality, lineage, security, governance, and compliance controls.
• Drive best practices in CI/CD, code quality, testing, release management, and operational support for data solutions.
• Contribute to platform improvements, architecture reviews, and technology evaluation activities.

Requirements:

• Bachelor's or master's degree in computer science, information technology, engineering, or a related field.
• 6-8 years of experience in data engineering, data platform engineering, or related roles.
• Strong hands-on experience with Azure cloud services and Azure Databricks.
• Deep expertise in Apache Spark / PySpark, Delta Lake, SQL, and distributed data processing.
• Strong understanding of data architecture, data modeling, and modern lakehouse patterns.
• Experience building and supporting complex ETL/ELT pipelines in enterprise environments.
• Working exposure to AI and ML data pipelines, feature preparation, or data support for AI-based solutions.
• Experience with batch and streaming data processing.
• Solid understanding of data governance, security, access control, and data quality practices.
• Experience with orchestration tools such as Azure Data Factory, Databricks Workflows, Airflow, or similar.
• Strong debugging, analytical thinking, and problem-solving skills.