Hadoop python Consultant

Position: Hadoop Python support Consultant

Work Model: Full-time Office (Mumbai)

Experience: 4+ Years

Availability: Immediate joiner

Shift Time: Rotational as below:

 Shift 1 : 6:30 AM - 3 PM

 Shift 2 : 2:30 PM - 11 PM

 Shift 3: 10:30 PM - 7 AM

Required Technical Skills (Mandatory)

• Python (Mandatory)

• Mainframe

• Hadoop Ecosystem

• Linux Administration

• Shell Scripting

• Apache Spark

• Sqoop

• Apache NiFi

• IBM MQ / MQ Series

• IBM DataStage

• SQL Concepts

• ITIL Processes (Incident, Change & Problem Management)

What You'll Do

• Provide 24*7 Level 1 and Level 2 support for enterprise data platforms and batch processing environments.

• Monitor and maintain Hadoop, Spark, NiFi, MQ, DataStage, and Mainframe applications.

• Develop and maintain Python and Shell scripts for automation and operational efficiency.

• Support data ingestion, transformation, and integration pipelines across multiple platforms.

• Troubleshoot production incidents and ensure timely resolution within defined SLAs.

• Perform scheduled maintenance, patching, and upgrades across data processing environments.

• Monitor system health, data flows, batch jobs, and integration processes.

• Collaborate with infrastructure, application, database, and business teams to resolve issues.

• Drive continuous improvement initiatives through automation and process optimization.

• Ensure adherence to operational, security, and compliance standards.

Key Responsibilities

• Provide day-to-day support for Hadoop, Spark, NiFi, MQ, and Mainframe environments.

• Monitor batch processing, data pipelines, and integration workflows.

• Analyze and resolve production issues, system failures, and performance bottlenecks.

• Develop Python-based utilities and Shell scripts to automate routine operational tasks.

• Manage job scheduling, execution monitoring, and failure recovery activities.

• Perform data validation, reconciliation, and troubleshooting activities.

• Support data migration and ingestion processes using Sqoop and related tools.

• Configure alerts, dashboards, and monitoring mechanisms for proactive incident management.

• Participate in release deployments, system upgrades, and patch management activities.

• Prepare Root Cause Analysis (RCA) documents for major incidents.

• Coordinate with cross-functional teams for issue resolution and service improvements.

• Maintain operational documentation, runbooks, and standard operating procedures.

• Participate in 24x7 support rotations.

• Ensure compliance with change management, incident management, and problem management processes.

Skills & Attributes

• Strong analytical and troubleshooting skills.

• Ability to work in complex production support environments.

• Experience in automation using Python and Shell scripting.

• Excellent communication and stakeholder management skills.

• Ability to prioritize and manage multiple tasks simultaneously.

• Team player with mentoring and knowledge-sharing capabilities.

• Focus on operational excellence and continuous improvement.

What Makes You a Qualified Candidate

• 4+ years of experience in Production Support

• Strong hands-on experience in Python programming.

• Experience working with Hadoop ecosystem components.

• Hands-on experience with Apache Spark for data processing.

• Strong Linux administration and Shell scripting knowledge.

• Experience with Apache NiFi for data flow management.

• Knowledge of Sqoop for data transfer between relational databases and Hadoop.

• Experience supporting IBM MQ / MQ Series environments.

• Hands-on experience with IBM DataStage development or support.

• Exposure to Mainframe applications and batch processing systems.

• Strong SQL and database troubleshooting skills.

• Experience with monitoring, performance tuning, and capacity planning.

• Understanding ETL concepts.

• Excellent verbal and written communication skills.

• 4–8 years of overall IT experience.

Preferred Qualifications

• Strong knowledge in Hadoop, Spark, Python, or Cloud Technologies.

• ITIL Foundation certification(Not Mandatory).

• Experience in Banking.

• Knowledge of Agile methodologies and enterprise support models.