Kafka

Responsibilities :

Design, implement, and support Kafka-based streaming solutions for high-throughput, low-latency workloads.

Configure and manage topics, partitions, replication, retention, and compaction policies aligned to application needs.

Ensure cluster reliability and availability through capacity planning, upgrades, patching, and proactive maintenance.

Troubleshoot production issues including consumer lag, broker performance, rebalancing, and message delivery concerns.

Define and enforce standards for naming, ACLs, quotas, and multi-tenant usage to ensure safe platform adoption.

Implement monitoring and alerting for Kafka health, throughput, latency, and storage using metrics and logs.

Partner with engineering teams to guide producer/consumer configurations and best practices for scalability and resilience.

Drive performance tuning across brokers and clients, including partition strategy and batching/compression settings.

Document operational runbooks, architecture patterns, and onboarding guidelines for consistent usage across teams.

Technical and Professional Requirements:

Technology- Cloud Platform- AWS Data Analytics- Amazon Managed Streaming for Apache Kafka