Lead Data Engineer

Role: Lead Data Engineer

Experience: 8+ Years

Location: Chennai | Pune | Coimbatore | Bangalore (Hybrid)

Job Summary

We are looking for an experienced Lead Data Engineer with deep expertise in the Google Cloud Platform ecosystem to design, build, and maintain scalable data processing systems. The ideal candidate should have strong hands-on experience with BigQuery, Dataflow, Airflow, and advanced SQL, alongside programming capabilities in Python or Java.

Key Responsibilities

• Design and develop scalable data pipelines on Google Cloud Platform.
• Build and optimize datasets using BigQuery and advanced SQL.
• Implement real-time and batch data processing using Dataflow.
• Implement workflow orchestration using Airflow / Cloud Composer.
• Write efficient, production-ready code using Python or Java.
• Optimize BigQuery queries and pipeline execution paths for performance and cost.
• Work closely with analytics, data science, and product teams.
• Ensure data quality, reliability, and governance standards.
• Troubleshoot and resolve pipeline failures in production.

Mandatory Skills

• Strong experience with the Google Cloud Platform (GCP) ecosystem.
• Hands-on expertise in BigQuery and advanced SQL.
• Experience with Dataflow for stream and batch data processing.
• Experience with Airflow (preferably GCP Cloud Composer).
• Strong coding skills in Python.
• Basic Knowledge in Java

Good to Have

• Experience with Dataform or dbt (data transformation tools).
• Experience with Dataproc or other GCP data services.
• Knowledge of GCP security, IAM, and data governance tools.
• Experience handling large-scale data and complex transformations.

Why Join Us?

• Join KANINIs award-winning Data Engineering Team, recognized as the "Outstanding Data Engineering Team" at DES 2025.
• Work on real-world problems at the intersection of data, AI, and product development.
• Be part of a collaborative, innovative, and growth-driven environment.
• Access to cutting-edge tools, leadership support, and continuous learning opportunities.