SDE - II Data Platform

SDE II - Data Platform - Lucknow

Mission

Contribute to building a self-serve Data Platform-as-a-Product enabling scalable, reliable, and easy-to-use data infrastructure across batch and streaming workloads.

Core Responsibilities

• Build and maintain scalable data pipelines (batch & streaming) using Spark/Flink
• Develop integrations across Airflow, Iceberg, and cloud platforms
• Apply best practices in data modeling, partitioning, and query optimization
• Support Kubernetes-based data platform workloads and troubleshoot issues
• Optimize pipelines for performance, cost, and reliability
• Implement monitoring, logging, and data quality checks
• Participate in on-call and incident resolution

Required Skills

• 3–6 years in data/platform engineering
• Strong fundamentals: SQL, data modeling, partitioning, optimization
• Hands-on with Spark (PySpark/Scala); exposure to Flink is a plus
• Experience with Airflow (or similar orchestration tools)
• Familiarity with data lake formats (Iceberg/Delta/Hudi)
• Working knowledge of Kubernetes & cloud (GCP/AWS/Azure)
• Programming: Python (must-have), Scala (good-to-have)
• Exposure to CI/CD, Git, and IaC (Terraform/Helm basics)

Good to Have

• Kafka / streaming systems experience
• DBT, Great Expectations, or similar tools
• Observability tools (Prometheus, Grafana)
• Metadata/catalog tools (OpenMetadata/DataHub)