Azure Data Engineer

Roles and Responsibilities :

• Design, develop, test, deploy and maintain large-scale data pipelines using Azure Data Factory (ADF) to extract, transform, load data from various sources into Azure storage solutions.
• Collaborate with cross-functional teams to gather requirements for data processing needs and design scalable architectures that meet business needs.
• Develop complex SQL queries to optimize database performance and troubleshoot issues in production environments.
• Ensure high availability of data systems by implementing monitoring tools, logging mechanisms, and alerting configurations.

Job Requirements :

• 5-8 years of experience in Data Engineering with expertise in ADF, Pyspark, Data Bricks.
• Strong understanding of SQL programming language for querying relational databases.
• Experience working on big-data technologies such as Hadoop ecosystem components including Hive & Pyspark.