Azure Data Engineer

Key Responsibilities

• Lead the design and implementation of enterprise-scale data platforms on Azure.
• Drive architecture, development, and optimization of data pipelines using Azure Databricks.
• Develop scalable ETL/ELT solutions using PySpark and Spark frameworks.
• Lead data ingestion, transformation, and processing for structured and unstructured data.
• Design Lakehouse architectures for enterprise reporting and analytics.
• Collaborate with business stakeholders, architects, and analytics teams to define data solutions.
• Ensure platform scalability, performance, security, and cost optimization.
• Establish Data Engineering best practices, coding standards, and CI/CD processes.
• Mentor and guide a team of data engineers and provide technical leadership.
• Support customer-facing discussions, solution design workshops, and architecture reviews.

Required Skills

• 6+ years of IT experience with strong Data Engineering background.
• Hands-on experience in Azure Databricks.
• Strong expertise in PySpark and Spark optimization techniques.
• Experience with Azure Data Factory (ADF).
• Strong knowledge of Azure Data Lake Storage (ADLS Gen2).
• Expertise in SQL, Data Modeling, and Data Warehousing concepts.
• Experience building enterprise-scale data pipelines and batch/streaming solutions.
• Knowledge of Delta Lake, Lakehouse architecture, and performance tuning.
• Experience with Git, Azure DevOps, and CI/CD implementation.
• Strong stakeholder management and communication skills.