Data Engineer

Roles and Responsibilities :

• Design, develop, test, deploy and maintain large-scale data pipelines using Scala and Spark.
• Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs.
• Ensure high scalability, reliability, and performance of the distributed system by implementing efficient algorithms and monitoring its behavior.
• Troubleshoot issues related to data processing, transformation, and distribution through debugging techniques.

Job Requirements :

• 4-5 years of experience in application development with a focus on big data technologies such as Scala/Spark.
• Strong understanding of data pipeline concepts including data transformations from various sources into target formats.
• Experience working with distribution systems for reliable data processing at scale is essential.