Big Data Engineer

Roles and Responsibilities :

• Design, develop, test, deploy and maintain large-scale data pipelines using Airflow.
• Collaborate with cross-functional teams to gather requirements and design solutions for complex data processing needs.
• Develop scalable and efficient algorithms in Java to process massive datasets stored in Dynamo DB.
• Troubleshoot issues related to BigQuery queries performance optimization.

Job Requirements :

• 3-5 years of experience in big data engineering with expertise in Spark.
• Strong understanding of SQL concepts including query optimization techniques.
• Experience working with cloud-based technologies such as AWS (Dynamo DB) is required.