Data Engineer

Roles and Responsibilities :

• Design, develop, test, deploy, and maintain large-scale data pipelines using Cloudera ecosystem.
• Collaborate with cross-functional teams to gather requirements and design solutions for complex business problems.
• Develop ETL processes to extract, transform, and load data from various sources into a centralized repository.
• Troubleshoot data quality, performance tuning, and scalability issues.
• Ensure data quality and integrity by implementing robust testing and validation procedures.
• Optimize data processing workflows for improved efficiency and performance.

Job Requirements :

• Strong proficiency in Python programming language with experience working on big-data frameworks like Spark.
• Experience with SQL querying languages such as HQL or equivalent.
• Familiarity with Cloudera or Informatica BDM is desirable.
• Ability to work collaboratively with cross-functional teams to gather requirements and design solutions.
• Strong problem-solving skills and attention to detail.
• Experience with data engineering tools and technologies is an asset.