Senior Data Engineer Kafka and Streaming Data Processing

Senior Data Engineer Kafka & Streaming Data Processing
Location: Pune

Experience: 10-12 Years

About the Role:
We're hiring a Senior Data Engineer to build and scale real-time data processing pipelines using Apache Kafka and Debezium on AWS and open source stack like Apache Iceberg, Open metadata, Atlas, Apache Airflow. If you're passionate about streaming architectures and enjoy working across the AWS data ecosystem and open source stack, this role is for you.

Responsibilities:
Design & manage Kafka-based streaming pipelines (topics, partitioning, consumer groups), Flink

Design & manage Spark based Batch pipelines using Spark based framework

Configure and maintain Debezium connectors for real-time CDC
Build scalable ETL/ELT workflows using PySpark and AWS Airflow

Strong experience in utilizing Iceberg tables and AWS open source system.

Manage and optimize AWS Redshift for analytics & reporting
Ensure pipeline reliability, data quality, and performance at scale
Collaborate with cross-functional teams on data requirements

What We're Looking For:
10-12 years of Data Engineering experience with strong hands-on Kafka, CDC, AWS skills (must-have)
Experience with Debezium / CDC tools
Proficiency in PySpark, SQL, AWS Airflow, Glue, Iceberg and Redshift
Working knowledge of AWS services (S3, IAM, Lambda)
Strong SQL and data modeling fundamentals

Nice to Have:
Experience in the Finance/Banking domain