Senior Data Platform Engineer (Hybrid Work Model)

Job Summary

QRR-4839 Senior Data Platform Engineer (Hybrid Work Model)

Location: Pune, India

Model of Work: Hybrid

Job Purpose

We are looking for a Senior Data Platform Engineer to join the OPTI (Office of Product and Technology Innovation) AI Platform team in Pune. You will build, deliver, and maintain the data pipelines, analytics, dashboards, and reporting that run on the Databricks platform of the OPTI AI team, and you will own their data quality and monitoring.

You are a strong individual contributor who takes ownership of complex technical problems, brings opinions to design discussions, and raises the quality bar of everything you touch. You wont just execute tickets: youll help shape how data moves, transforms, and gets consumed. You will work day to day with the Pune and US, AI Platform team and the team supports what it builds in production.

What You Will Do

• Design, build, and maintain scalable data pipelines that move and transform data across Bronze, Silver, and Gold layers in a Medallion Architecture
• Develop and optimize data workflows in Databricks, Databricks Workflows, and Unity Catalog
• Build dashboards and reports with stakeholders/SMEs, turning their requirements into reliable, well-designed views of the data
• Own data quality and monitoring for your pipelines: checks, alerting, and observability that keep SLAs met consistently
• Take part in design reviews, bringing practical experience and a critical eye to technical decisions
• Use AI coding tools and our AI Development Life Cycle (AIDLC) as part of how you build, test, and ship

• Work with Azure data services for storage, orchestration, security, and infrastructure, alongside the teams DevOps engineer
• Collaborate with stakeholders, AI Platform team in US and India to shape what gets built and why
• And other duties as assigned.

What to Bring

Data engineering fundamentals

• 5+ years of hands-on data engineering experience in a production environment
• Strong Python and SQL, complex joins, and performance tuning
• Solid understanding of pipeline design patterns: incremental loading, schema evolution, and error handling
• Experience with multiple ingestion methods: Batch based, JDBC, API driven, KAFKA etc

Databricks and Spark

• Hands-on experience with Apache Spark or other large-scale data processing tools
• Databricks experience strongly preferred, Unity Catalog, and Databricks Workflows
• Working knowledge of Medallion Architecture and how to structure Bronze, Silver, and Gold layers for different data domains and Star-schema presentation layer. Useful knowledge of SCD2, data lineage, data quality measures, etc

Quality, testing, and observability

• A testing mindset: automated tests for transformations, data quality checks, and validation built into the pipeline
• Experience setting up monitoring and alerting for production pipelines, and debugging and optimizing slow or failing ones
• Familiarity with CI/CD for data pipelines, including version control, automated testing, and deployment (GitHub Actions or Azure DevOps)

Reporting

• Strong Experience building dashboards and reports that business users rely on

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.