Data Engineer

Roles and Responsibilities :

• Design, develop, test, and deploy large-scale data pipelines using Hadoop, Spark, and Snowflake.
• Collaborate with cross-functional teams to gather requirements and deliver high-quality data solutions.
• Develop complex SQL queries to extract insights from massive datasets stored in Parquet format on AWS S3.
• Troubleshoot issues related to data processing, storage, and retrieval.

Job Requirements :

• 3-6 years of experience in Data Engineering with expertise in Hadoop ecosystem (HDFS) and Spark.
• Strong proficiency in Python programming language with knowledge of AWS services such as S3, Glue, Lambda etc. .
• Experience working with relational databases like Snowflake is required; strong understanding of SQL concepts including joins, aggregations, filtering etc. .
• Familiarity with Parquet file format for storing structured data.