Hadoop Administrator

Candidate should have hands-on exposure on given skills and is willing to work at client office 3 days/week

• BigData / Hadoop Administration
• Programming & Scripting: Linux, SQL
• Data Engineering & Processing: Apache NiFi, Kafka, Iceberg, Trino, Starburst
• Big Data Ecosystem: HPE Ezmeral, HDFS, Yarn RM
• Search & Analytics: ElasticSearch
• Storage & Infrastructure: MinIO, Kubernetes
• CI/CD & Collaboration Tools: ServiceNow, Jira, GitHub, ArgoCD, Copilot

Roles & Responsibilities

Roles & Responsibilities:

• Managing HDFS storage, disk utilization and data availability to maintain cluster health mainly HPE Ezemeral platform.
• Managing security-related activities, including service accounts password resets and certificate renewals
• Supporting Hive and Big Data workloads, troubleshooting job failures for downstream NiFi ingestion like to ssh failures/krb issues.
• Supporting and performing HPE Ezemeral server reboots and ensure high availability of the system during such maintenance activities
• Monitoring YARN ResourceManager/NodeManager and cluster resource utilization, troubleshooting application and resource-related issues.
• Manage end-to-end production incident and change management, including monitoring, troubleshooting, RCA, and SLA adherence.
• Manage and maintain Apache NiFi data ingestion workflows across multiple environments, ensuring reliable pipeline execution and resolving operational issues.
• Support and troubleshoot Kubernetes-based data platforms, ensuring cluster stability, availability, and performance.
• Manage Kafka streaming pipelines, including monitoring, troubleshooting data issues, consumer lag, and pipeline backlogs.
• Manage Trino/Starburst and Iceberg-based data lakehouse platforms
• Manage MinIO object storage supporting data lake and ingestion pipelines.
• Support SIEM data management
• Manage CI/CD deployments and configuration management using GitHub and ArgoCD across environments.
• Perform platform maintenance and security operations, certificate/svc ID renewals, authentication-related activities, and scheduled maintenance.
• Monitor platform and pipeline health using Grafana and Elastic/Kibana, proactively identifying and resolving operational issues.
• Support platform/data migrations, including environment assessment, dependency mapping, documentation.