Data Engineer

Roles and Responsibilities :

• Design, develop, test, deploy and maintain large-scale data pipelines using Airflow to extract insights from Snowflake databases.
• Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs.
• Develop scalable and efficient ETL processes using PySpark on AWS Glue to integrate various data sources into the pipeline.
• Troubleshoot issues related to pipeline failures or performance problems by analyzing logs, debugging code snippets.

Job Requirements :

• 5-8 years of experience in Data Engineering with expertise in Airflow, Snowflake, and Python (PySpark).
• Strong understanding of big data technologies such as Hadoop ecosystem components including HDFS, MapReduce, Spark Core & MLlib.
• Experience working with cloud-based platforms like AWS Glue for building ETL pipelines.
• Willing to work in a startup company