Lead Data Engineer

Job Summary

Lead the design, build, and operations of a production-grade data platform. Own end-to-end pipelines, set engineering standards, and mentor a team of Data Engineers delivering data for analytics, ML, and product use cases.

Experience: 5-8 years | Location: Bangalore/India | Type: Full-time

Responsibilities

• Lead mentor a team of Data Engineers - code reviews, design reviews, growth plans
• Design and operate ETL/ELT pipelines in Python for batch and streaming workloads
• Own data modelling across PostgreSQL / Supabase (OLTP) and Cassandra (wide-column)
• Build and run orchestration on Apache Airflow - DAG design, SLAs, retries, backfills
• Deploy and operate the platform on AWS (EC2-based, self-managed stack)
• Establish data governance: catalog, lineage, ownership, MDM, data quality, profiling, observability
• Drive engineering excellence - Agile delivery, CI/CD, testing, IaC, code review

Product Mindset

• Own outcomes across the full lifecycle - design, delivery, operations, iteration
• Optimise for scale, reliability, and maintainability with measurable SLAs and cost targets
• Prioritise based on user value (analytics, ML, product teams as customers)
• Ship iteratively; measure adoption, quality, and reliability continuously

Must-Have

• 5-8 years in data engineering with proven experience leading and mentoring engineers
• Expert Python and strong SQL
• Production experience with PostgreSQL, Supabase, and Apache Cassandra
• Hands-on ownership of Apache Airflow at scale
• Comfortable running data workloads on AWS EC2 (Linux, networking, self-managed services)
• Strong ETL, data modelling, orchestration fundamentals
• Practical experience implementing data governance - MDM, data quality, data profiling, observability, lineage/catalog
• Agile, CI/CD, and coding best practices as everyday habits

Good-To-Have

• Data security and compliance - encryption (KMS/at-rest/in-transit), PII handling, GDPR/PDPA
• Streaming (Kafka, Kinesis) and lakehouse (Iceberg, Delta) exposure
• IaC (Terraform, Ansible) for self-managed AWS stacks

Education

Bachelor's or Master's in Computer Science, Engineering, or related field.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.