Data Engineer

Roles and Responsibilities

• Design, develop, and maintain large-scale data pipelines using Python and PySpark.
  
• Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions.
  
• Develop scalable data warehousing solutions on AWS cloud platform.
  
• Ensure data quality by implementing robust testing frameworks and monitoring dashboards.
  
• Participate in code reviews to ensure adherence to coding standards.
  

Desired Candidate Profile

• 4-9 years of experience in Data Engineering with expertise in Data Warehousing.
  
• Strong proficiency in Python programming language with hands-on experience working with PySpark.
  
• Experience designing and developing ETL processes for large datasets using AWS services such as S3, Lambda, Step Functions etc. .
  
• Knowledge of SQL concepts including query optimization techniques for performance tuning.