Lead Data Engineer

We are seeking an experienced Lead Data Engineer with 7+ years of hands-on experience in SQL Server, Azure Data Factory (ADF), ETL/Data Integration, and Azure Data Platform services. The ideal candidate will be responsible for designing, developing, optimizing, and supporting enterprise-scale data pipelines and cloud-based data solutions.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory and Azure services.
• Develop and optimize complex T-SQL queries, stored procedures, functions, views, and database objects in SQL Server.
• Perform SQL Server database development and administration, including performance tuning, troubleshooting, and data integrity management.
• Build and maintain data integration pipelines across multiple source and target systems.
• Develop and support cloud-based data solutions using Azure data platform technologies.
• Analyze source systems and evolving data structures, adapting pipelines to meet business requirements.
• Apply data modeling techniques to support analytical and operational reporting needs.
• Work with large volumes of structured and semi-structured data to ensure efficient processing and performance.
• Troubleshoot data pipeline failures, data quality issues, production incidents, and performance bottlenecks.
• Monitor data pipelines and proactively resolve issues.
• Collaborate with architects, developers, QA teams, business analysts, and stakeholders to deliver robust data solutions.
• Participate in Agile ceremonies including sprint planning, stand-ups, refinements, reviews, and retrospectives.
• Maintain technical documentation covering data flows, mappings, dependencies, and operational procedures.
• Participate in code reviews and follow development, security, and deployment best practices.
• Support production deployments, releases, and post-production activities.

Required Skills

• Strong experience in SQL Server Development and Performance Tuning.
• Hands-on expertise with Azure Data Factory (ADF).
• Experience in ETL/ELT development and Data Integration.
• Strong knowledge of Azure Data Platform Services.
• Advanced T-SQL programming skills.
• Experience working with structured and semi-structured data.
• Knowledge of data modeling concepts.
• Strong debugging and troubleshooting skills.
• Experience working in Agile development environments.

Preferred Skills

• Exposure to Azure Synapse Analytics, Azure Data Lake, or Databricks.
• Knowledge of CI/CD and DevOps practices.
• Experience in enterprise-scale data warehousing projects.

Qualifications

• Bachelor's degree
• 7+ years of experience in Data Engineering and Data Integration.