Lead Data Engineer

Job Summary

Senior Data Engineer (Senior Individual Contributor / Technical Lead).

Location Bengaluru; Minimum Experience 6 to 8 years.

Responsibilities

• Design and deliver scalable pipelines.
• Build batch and near-real-time ETL/ELT pipelines using SQL, Python and PySpark, with Snowflake as a primary cloud data platform.
• Engineer reliable data models.
• Create dimensional, normalized and analytical models that support efficient querying, traceability and reuse across enterprise reporting and analytics.
• Own data quality by design. Implement automated profiling, schema checks, reconciliation, completeness, freshness, uniqueness and business-rule validation; publish quality results and route failures for action.
• Build CI/CD for data products. Create and maintain automated build, test and deployment pipelines for SQL, dbt, orchestration code and infrastructure configuration, including version control, peer review, environment promotion and safe rollback.
• Strengthen observability and operations. Monitor pipeline health, lineage, cost, performance and service-level objectives; troubleshoot incidents, perform root-cause analysis and eliminate recurring manual support.
• Integrate diverse sources. Ingest and standardize data from databases, APIs, files and event sources, applying secure access patterns and appropriate handling for sensitive information.
• Modernize legacy workflows. Migrate or refactor Informatica PowerCenter and other legacy ETL processes into maintainable cloud-native patterns using tools such as dbt and modern orchestration platforms.
• Partner from requirement to adoption. Translate stakeholder needs into technical designs, validate production outcomes and communicate tradeoffs, risks and delivery progress clearly.
• Lead through engineering practice. Set coding and testing standards, review designs and pull requests, mentor engineers and coordinate delivery across cross-functional teams.

Qualifications

Required Qualifications

• Bachelors or masters degree in computer science, engineering, mathematics or a related discipline, or equivalent practical experience.
• Substantial professional experience designing and operating enterprise data pipelines and data warehouses in production.
• Advanced SQL skills and strong hands-on experience with Python and/or PySpark for data processing and automation.
• Deep experience with Snowflake, including performance optimization, security, workload management and scalable data modeling.
• Experience building ETL/ELT solutions with dbt, Informatica PowerCenter or comparable transformation and integration tools.
• Hands-on experience implementing CI/CD pipelines for data engineering workloads using Git-based workflows and automated testing/deployment tools.
• Experience building automated data quality systems with measurable rules, ing, lineage and issue-resolution workflows.
• Strong grounding in data profiling, cleansing, standardization, reconciliation and production data validation.
• Ability to gather ambiguous requirements, make sound design decisions and communicate effectively with technical and non-technical partners.
• B Tech / M Tech

Preferred Qualifications

• Snowflake SnowPro Core or comparable cloud data certification.
• Experience with REST API, event and/or streaming data, and high-volume security or operational datasets.
• Familiarity with data observability and quality frameworks such as dbt tests, Great Expectations, Soda or Deequ.
• Experience delivering data solutions in regulated, security-sensitive, healthcare, telecommunications or audit-oriented environments.
• Demonstrated technical leadership, mentoring or delivery coordination across a small engineering team.

Skills

• Snowflake
• SQL
• Python
• PySpark
• dbt
• ETL/ELT
• data modeling
• Git
• REST API

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.