Senior Data Engineer

Roles and Responsibilities :

• Design, develop, test, deploy and maintain large-scale data pipelines using DSA (Data Science Automation) tools.
• Collaborate with cross-functional teams to gather requirements and deliver high-quality data solutions that meet business needs.
• Develop complex SQL queries to extract insights from large datasets using PySpark and Python scripts.
• Troubleshoot issues related to data processing, storage, and retrieval in a distributed computing environment.

Job Requirements :

• 6-13 years of experience in Data Engineering with expertise in DSA technologies such as Airflow, Spark, etc.
• Strong proficiency in Python programming language with experience working with libraries like NumPy, Pandas, Matplotlib.
• Experience designing scalable architectures for handling big-data workloads using cloud-based platforms like AWS or GCP.