App Dev & Support Engineer II

Job Description:

Key Responsibilities

Production Support (L2/L3): Monitor, troubleshoot, and resolve production job failures (Oozie, Spark), data ingestion errors, and system slowness in a 24/7 environment.

Minimum 3 to 5 Yrs of experince is required

• Apache NiFi Administration: Install, configure, and maintain Apache NiFi/NiFi Registry. Design, optimize, and troubleshoot complex data flows and back-pressure settings.
• Kafka Management: Manage Apache Kafka clusters, including Kafka Connect, Schema Registry, and topic management. Perform rebalancing, consumer lag investigation, and message serialization
• Hadoop Ecosystem Support: Monitor HDFS, YARN, Hive, and Spark jobs. Maintain HBase, Sqoop, and ZooKeeper for reliability.troubleshooting.
• Root Cause Analysis (RCA): Perform deep-dive analysis on incident reports to establish root causes and implement permanent fixes, reducing future downtime.
• Performance Tuning: Tune NiFi flows, Kafka producers/consumers, and Spark/Hive queries to enhance performance.
• Deployment and Upgrades: Collaborate with DevOps to implement CI/CD processes for deploying NiFi flows and Big Data application artifacts