Lead Data Engineer

We are Hiring! Lead Data Engineer | Pune - Onsite

Location: Pune (Full-time)

Experience: 8+ Years

Skills: Python, SQL, Databricks, Spark/Pyspark, ETL

Role Overview:

As a Lead Data Engineer, you will drive the design and development of scalable batch & streaming data pipelines, work with modern Lakehouse/Data Warehouse platforms, and partner with analytics and business teams to enable data-driven decision-making.

Key Responsibilities:

• Data Pipeline Development: Design, build, and maintain high-performance batch and streaming pipelines using PySpark/Scala and Databricks.
• Data Modeling & Architecture: Design efficient schemas and curated datasets optimized for speed, scalability, and cost across Lakehouse/Data Warehouse platforms.
• Data Integration: Seamlessly integrate data from APIs, databases, flat files, and streaming sources across cloud platforms.
• Performance Optimization: Identify bottlenecks, tune Spark configurations, and optimize queries for large-scale data processing.
• Data Quality & Governance: Establish data validation checks and quality standards to maintain reliable data platforms.
• Collaboration & Leadership: Partner with Data Scientists, Analysts, and Cloud Engineers while setting technical best practices, version control, and CI/CD standards.

Key Qualifications & Requirements:

• Education: Bachelor's degree in Computer Science, Engineering, or a related field.
• Experience:8+ years of hands-on experience in Data Engineering.
• Technical Skills:
• Strong expertise in Python, Scala, or Java and high proficiency in SQL.
• Deep experience with Apache Spark (PySpark/Scala) and Databricks.
• Familiarity with cloud platforms (AWS, Azure, or GCP) and data warehouses (Snowflake, Redshift, etc.).
• Sound knowledge of data modeling principles, ETL design patterns, and big data frameworks.