Hadoop Admin

Educational Requirements Service Line Responsibilities

• Cluster Management
• Install, configure, and maintain Hadoop clusters (Cloudera, Hortonworks, or Apache Hadoop)
• Manage core components including HDFS, YARN, MapReduce
• Perform cluster scaling, upgrades, and patching
• Administration Monitoring
• Monitor cluster performance, health, and usage
• Troubleshoot and resolve cluster issues, failures, and bottlenecks
• Implement alerting and monitoring tools (Ambari, Cloudera Manager, Grafana)
• Security Governance
• Configure and manage Kerberos authentication
• Implement data security policies using Ranger / Sentry
• Handle user access controls and permissions
• Data Management
• Manage data storage, replication, and backup strategies
• Optimize HDFS performance and storage utilization
• Handle data ingestion and integration workflows
• Performance Optimization
• Tune Hadoop cluster performance (memory, CPU, storage)
• Optimize YARN resource allocation and scheduling
• Identify and resolve performance issues in jobs
• Integration Support
• Support tools in Hadoop ecosystem: Hive, Spark, HBase, Sqoop / Flume / Kafka
• Work closely with data engineers and developers

Additional Responsibilities

• Experience with cloud-based Hadoop (AWS EMR, Azure HDInsight, GCP Dataproc)
• Knowledge of DevOps practices (CI/CD, Docker, Kubernetes)
• Experience in high availability (HA) and disaster recovery (DR) setups

Technical and Professional Requirements

• Core Technical Skills: 36 years of experience in Hadoop Administration; Strong knowledge of HDFS, YARN, MapReduce; Hadoop ecosystem tools; Experience with Linux system administration (mandatory)
• Big Data Ecosystem Familiarity: Hive, Spark, HBase, Kafka, Sqoop, Flume; Understanding of distributed computing concepts
• Tools Platforms: Cloudera Manager / Ambari; Monitoring tools (Nagios, Prometheus, Grafana); Basic scripting in Shell / Python

Preferred Skills

• Technology- >Big Data - Hadoop- >Hadoop Administration