Data Lake Developer

Roles and Responsibilities :

• Design, develop, test, deploy and maintain large-scale data pipelines using Airflow.
• Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions on time.
• Develop complex data workflows using Python, PySpark, and other technologies to process large datasets from various sources.
• Troubleshoot issues related to data processing, ETL processes, and database queries.

Job Requirements :

• 5-15 years of experience in Data Engineering with expertise in Airflow development.
• Strong proficiency in Python programming language with experience working with libraries such as Pandas, NumPy, Matplotlib etc. .
• Experience working with big data technologies like Spark (PySpark) for distributed computing tasks.
• Familiarity with version control systems like Git for managing code repositories.