4+ yrs exp. in Python/Scala data pipeline development and support

Experience: 4+ Years

Role: Senior Data Engineer

Employment Type: Full-Time

Work Location:  Mumbai, Chennai and Bangalore

Job Summary

We are looking for an experienced Senior Data Engineer with strong expertise in Python, Java, or Scala, SQL, and modern data engineering frameworks. The candidate will be responsible for designing, developing, optimizing, and maintaining scalable data pipelines for batch and streaming data processing.

The ideal candidate should have hands-on experience with technologies such as Apache Spark, Kafka, Flink, dbt, or equivalent frameworks, along with strong database knowledge, data modeling skills, and experience optimizing pipelines for performance and cost.

Experience with containerization, orchestration, and observability tools is expected. Familiarity with Microsoft Fabric Notebooks and Datadog will be an added advantage.

Key Responsibilities

• Design, develop, and maintain scalable and reliable data pipelines using Python, or Scala.
• Build and manage batch and streaming data processing solutions to support business and analytics requirements.
• Develop data integration, transformation, and processing workflows using Apache Spark, Kafka, Flink, dbt, or equivalent technologies.
• Work with SQL and NoSQL databases to support efficient data storage, retrieval, and processing.
• Apply data modeling principles to develop efficient and scalable data architectures.
• Optimize SQL queries, data pipelines, and storage to improve performance and reduce operational costs.
• Implement data quality checks, validation, error handling, and reconciliation processes.
• Use Docker to containerize applications and data processing workloads.
• Work with Kubernetes or equivalent orchestration platforms to deploy and manage workloads.
• Implement monitoring, logging, alerting, and observability for data pipelines using Datadog or similar tools.
• Troubleshoot production issues, pipeline failures, data inconsistencies, and performance bottlenecks.
• Collaborate with data architects, analysts, data scientists, and other engineering teams.
• Follow best practices for coding standards, version control, testing, documentation, and CI/CD.
• Explore and use Microsoft Fabric Notebooks for data engineering workflows where applicable.

Mandatory Skills

• 4+ years of experience in data engineering, big data development, or a related technical role.
• Strong programming experience in Python, or Scala.
• Hands-on experience with one or more modern data frameworks, such as Apache Spark, Kafka, Flink, dbt, or equivalent technologies.
• Strong SQL skills, including complex queries, joins, aggregations, and query optimization.
• Good understanding of SQL and NoSQL databases and data modeling principles.
• Experience developing batch and/or streaming data pipelines.
• Proven ability to optimize data pipelines, SQL queries, and storage for performance and cost.
• Experience with Docker and working knowledge of Kubernetes or equivalent orchestration tools.
• Experience with monitoring, logging, troubleshooting, and production support of data pipelines.

If this opportunity sounds interesting to you, Kindly share updated CV to [email removed] with below mention details. Once your profile is shortlisted you will receive the call from us

• Total Experience in Python-
• How Many Scala-
• Experience in data frameworks (Spark, Kafka, Flink, dbt, or equivalent) -
• Experience Sql -
• Experience in orchestration tools (Name) -
• Current CTC -
• Expected CTC -
• Notice Period/LWD -
• Offer if any -
• Current Location -
• Preferred Location-
• Can you take interview on 28th Oct 2026 -

Regards

Vineet Kumar