ETL Developer

Job Summary

Sr. Developer - ADF, SQL, Azure Synapse (8-10 years experience)

We are seeking a highly skilled and experienced senior member in team for development and management of ETL processes. The ideal candidate will have 8-10 years of experience in data engineering and ETL development. Hands on experience on Azure Data Factory (ADF), Synapse Pipelines, Azure Data Lake, SQL, PySpark, Synapse Spark Pools, Azure Data Lake Storage Gen2 (ADLS), Delta Lake, Azure Synapse Analytics (Dedicated SQL Pool, Serverless SQL), Data Vault. This role requires a strong technical background, hands-on expertise in ETL development, and the ability to drive business discussions and translate complex business logic into effective data solutions.

Key Responsibilities:

• Hands-on ETL Development Reporting Implementation:
  • Actively participate in the development, design, and implementation of ETL workflows using Azure Data Factory (ADF), Synapse Pipelines, Azure Data Lake, SQL, PySpark, Synapse Spark Pools
  • Working knowledge on Azure Data Lake Storage Gen2 (ADLS), Delta Lake, Azure Synapse Analytics (Dedicated SQL Pool, Serverless SQL), Data Vault
  • Hands-on development of complex ETL pipelines, data transformation logic, and integration processes to ensure efficient data flow and high-volume data processing.
  • Implement strategies for efficient data extraction, transformation, and loading from source to destination systems, ensuring robust and scalable solutions.
  • Oversee the setup, configuration, and deployment of ETL systems to ensure optimal performance.
  • Monitor ETL workflows, troubleshoot issues, and optimize data processing to handle high-volume data efficiently.
  • Continuously review and refine ETL pipelines to enhance performance, scalability, and fault tolerance.
  • Excellent knowledge in creating stored procedure, views, function etc.
• Business Logic Identification Implementation:
  • Work closely with business stakeholders , architects and team members to identify complex business logic and translate it into ETL processes, ensuring the accurate and effective transformation of data according to business requirements.
  • Design and implement business rules, data validation logic, and complex transformations within ETL workflows to ensure data integrity and alignment with business objectives.
• Quality Assurance Documentation:
  • Ensure adherence to best practices for ETL design, coding standards, and data governance.
  • Oversee code reviews and testing of ETL solutions to maintain high-quality standards.
  • Document ETL designs, business logic, and workflow processes to ensure knowledge sharing and maintainability of systems.

Key Skills Experience Required:

• Experience:
  • 8-10 years of experience in data engineering and ETL development. Hands on experience on Azure Data Factory (ADF), Synapse Pipelines, Azure Data Lake, SQL, PySpark, Synapse Spark Pools, Azure Data Lake Storage Gen2 (ADLS), Delta Lake, Azure Synapse Analytics (Dedicated SQL Pool, Serverless SQL), Data Vault
  • Proven experience in designing, developing, and deploying ETL workflows using Devops CICD pipelines and processes for high-volume, complex data environments.
  • Extensive hands-on experience with high-performance data processing and data integration strategies.
• Technical Skills:
  • Strong hands-on experience with Azure Data Factory (ADF) and Synapse pipeline setup, including building and managing pipelines for large-scale ETL processes with very good knowledge in Data warehousing
  • Advanced proficiency in SQL, including query optimization, complex data transformations, and working with large datasets.
  • Experience implementing complex business logic in ETL processes, including data validation, transformation, and enrichment.
  • Experience of cloud data storage solutions such as Azure SQL Database, Azure Data Lake, and Azure Blob Storage.
  • Experience of ETL tools and technologies like Azure Synapse Analytics, PySpark, Synapse Spark Pools, SSIS (good to have)
  • Experience working on Azure Data Lake Storage Gen2 (ADLS), Delta Lake
  • Experience with performance tuning, troubleshooting, and optimizing ETL pipelines to manage high-volume data.
• Soft Skills:
  • Excellent communication and interpersonal skills to drive business discussions and collaborate effectively with both technical and non-technical stakeholders.
  • Strong problem-solving ability, capable of tackling complex data engineering challenges and implementing efficient solutions.
  • Ability to handle multiple priorities, adapt to changing requirements, and meet deadlines.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.