Big Data Engineer

Key Requirements:

• Strong hands-on experience in PySpark, Python and Spark Core
• Strong knowledge of Advanced SQL
• Experience with Hadoop and Hive
• Good experience in Unix/Shell scripting
• Experience in building and managing data pipelines and ETL processes
• Strong understanding of Airflow, scheduling and orchestration
• Experience with incremental data processing, data partitioning and duplicate handling
• Exposure to Cloud and/or On-Premise data environments
• Understanding of SDLC, code migration, version control and deployment
• Experience handling production issues, performance challenges and long-running jobs
• Good stakeholder management and communication skills

What we are looking for:

A technically strong professional who is hands-on with coding and data engineering, and can independently manage the complete journey from development testing UAT deployment production support.