Lead Data Engineer

Role & Responsibilities

• Lead the design, development, and implementation of scalable Azure data engineering solutions.
• Design and develop high-volume ETL/ELT pipelines using Azure Data Factory, Azure Synapse, Databricks, PySpark and SQL.
• Build and optimize data lakes, data warehouses, Azure SQL and Synapse Dedicated SQL Pools.
• Design robust data models, datasets and analytical solutions to support business and reporting requirements.
• Develop data ingestion and migration solutions from legacy/on-premise systems to Azure.
• Handle large-scale and petabyte-level data processing with a focus on performance, reliability and scalability.
• Monitor, troubleshoot and optimize data pipelines, queries and data-processing workloads.
• Implement data quality, security, access control, governance and compliance standards.
• Collaborate with Product, Marketing, BI, Data Science and ML teams to understand requirements and deliver data solutions.
• Support Power BI, Tableau and real-time/offline analytics initiatives.
• Provide technical leadership, conduct code/design reviews, and establish engineering best practices.
• Mentor and guide junior and senior data engineers, supporting their technical growth.
• Participate in recruitment and help build and grow the Data Analytics/Data Engineering team.
• Drive continuous improvement, automation and operational excellence across the data platform.

Preferred Candidate Profile

• 7+ years of experience in Data Engineering with strong Azure cloud expertise.
• Strong hands-on experience with Azure Data Factory, Azure Synapse, Azure Databricks, Azure SQL and Azure Data Lake.
• Excellent knowledge of PySpark, SparkSQL and SQL.
• Strong experience in ETL/ELT, data pipelines, data migration, data modelling, data warehousing and data lakes.
• Experience with Synapse Dedicated SQL Pool / Azure SQL Data Warehouse and large-scale data processing.
• Experience handling high-volume event data and petabyte-scale datasets is highly desirable.
• Good knowledge of SSIS, Azure Analysis Services, Power BI and Tableau.
• Strong understanding of data security, RBAC, access management, governance and data quality.
• Experience with performance tuning, monitoring and troubleshooting production data pipelines.
• Proven ability to lead, mentor and guide data engineering teams.
• Strong communication and stakeholder-management skills with experience working across Product, BI, Analytics, Data Science and ML teams.
• Experience working in Agile/Scrum environments.
• Strong analytical, problem-solving and solution-design capabilities.
• Exposure to modern data platform/Data Fabric architecture is an added advantage.