Data Engineer

Project Role : Data Engineer

Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems.

Must have skills : Apache Kafka

Good to have skills : NA
Minimum 2 year(s) of experience is required

Educational Qualification : 15 years full time education

Job Title
Kafka Developer (Apache Kafka / Streaming Engineer) Design, develop, and maintain scalable Kafka-based data pipelines and real-time streaming applications
Build and manage Kafka producers, consumers, topics, partitions, and consumer groups
Monitor, tune, and optimize Kafka cluster performance, throughput, and latency
Troubleshoot and resolve issues related to data processing, message delivery, and system integrations
Collaborate with application teams, data engineers, and DevOps teams to deliver robust integration solutions
Develop and maintain Kafka Streams, KSQL, and Kafka Connect (source/sink connectors) implementations
Ensure high availability, reliability, fault tolerance, and data consistency across Kafka platforms
Implement security configurations, schema management (Avro/Protobuf), and data governance standards
Create and maintain technical documentation for Kafka configurations, workflows, and architecture
Provide production support, defect analysis (RCA), and performance tuning for Kafka-related applications
Perform root cause analysis (RCA) for Kafka-related defects and integration failures
Manage topic creation, schema evolution, and versioning strategies
Work on event-driven architecture and microservices integration
Qualifications
Bachelor s degree in computer science / engineering / IT or related field
35 years of experience in Kafka or real-time data streaming roles
Preferred Experience
Experience in event-driven architecture and microservices design
Exposure to CI/CD pipelines and DevOps tools (Docker, Kubernetes)
Experience with monitoring tools like Prometheus, Grafana
Experience in high-volume transaction systems or telecom/finance domains
Key Deliverables
Stable and scalable Kafka pipelines
Low-latency data streaming solutions
Effective troubleshooting and defect resolution
Documentation and knowledge sharing
Work Environment
Agile development setup with cross-functional teams
High dependency on integration systems (APIGEE, BWCE, microservices, etc.)


Qualification 15 years full time education