Data Engineer

Roles and Responsibilities

• Design, develop, and maintain large-scale data pipelines using Azure Databricks, Python, and Data Factory.
  
• Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions on time.
  
• Develop complex SQL queries to extract insights from massive datasets stored in Azure Blob Storage.
  
• Implement data quality checks and validation processes to ensure accuracy and integrity of the data.
  
• Troubleshoot issues related to pipeline failures or errors in real-time.
  

Desired Candidate Profile

• 9-14 years of experience as a Data Engineer with expertise in Azure Databricks, Python, and Data Factory.
  
• Strong understanding of big data technologies such as Hadoop Distributed File System (HDFS), Apache Spark, etc. .
  
• Experience working with cloud-based platforms like AWS or GCP is an added advantage.
  
• Proficiency in writing complex SQL queries for querying large datasets stored in Azure Blob Storage.