Test Lead - ETL QA / Data Engineering Testing

Job Summary

Experience: 6 - 9 Years
Location: Pune (Hybrid) 3 Days WFO
Employment Type: Full-Time

We are seeking an experienced Test Lead - ETL QA / Data Engineering Testing to lead the quality assurance strategy for modern data platforms and ETL/ELT pipelines. The ideal candidate will have strong expertise in Data Pipeline Testing, Databricks, SQL, Test Automation, and Data Quality Validation, along with proven leadership experience managing QA teams and driving testing best practices across data engineering initiatives.

Key Responsibilities

• Lead QA strategy, test planning, and execution for Data Engineering, ETL/ELT, and Databricks-based projects.
• Design and execute data validation, transformation, reconciliation, and data quality testing.
• Develop and maintain automated testing frameworks for data pipelines and workflows.
• Validate Databricks notebooks, Apache Spark jobs, and Delta Lake implementations.
• Perform SQL-based data validation and end-to-end testing across data platforms.
• Conduct performance, scalability, and load testing for data processing systems.
• Lead and mentor QA team members while establishing testing standards and best practices.
• Drive test automation and CI/CD integration to improve delivery quality.
• Collaborate with Data Engineers, Architects, and Business teams to define testing requirements.
• Validate data lineage, data governance, and compliance requirements.
• Track and report data quality metrics, defect trends, test coverage, and release readiness.

Required Skills & Experience

Must Have

• 6+ years of experience in QA, Software Testing, or Quality Assurance.
• Minimum 2+ years of hands-on experience testing Data Engineering projects, ETL pipelines, or data platforms.
• 5+ years of experience leading QA teams or testing functions.
• Strong expertise in Databricks, Apache Spark, or ETL tools such as Informatica, Talend, AWS Glue, or Azure Data Factory.
• Advanced SQL skills for data validation, reconciliation, and testing.
• Strong understanding of Data Warehousing, ETL/ELT processes, and Data Pipeline architectures.
• Experience building and implementing Data Quality frameworks.
• Hands-on experience with Test Automation using Python, Java, or Scala.
• Knowledge of CI/CD tools such as Jenkins, GitHub Actions, or GitLab CI.
• Understanding of Data Governance, Metadata Management, and Compliance Testing.
• Strong leadership, stakeholder management, and communication skills.

Good to Have

• Experience testing real-time or streaming data platforms (Kafka, Event-driven Architectures).
• Exposure to Machine Learning and AI data pipelines.
• Experience with cloud platforms like AWS, Azure, or GCP.
• API testing experience using tools such as Postman or REST Assured.
• Hands-on experience with Data Quality tools such as Great Expectations or dbt.
• Exposure to monitoring and observability tools like Grafana, Datadog, or Splunk.
• Knowledge of Docker and Kubernetes.
• Relevant certifications such as ISTQB, Databricks Certified Associate, AWS Data Analytics, Azure Data Engineer, or GCP Data Engineer.
• Experience in data-intensive domains such as Banking, Financial Services, Healthcare, or Retail.

Mandatory Skills

• Data Pipeline Testing
• ETL Testing
• Databricks
• Apache Spark
• SQL
• Test Automation
• Data Quality & Validation
• Python / Scala
• Git & Version Control
• CI/CD Integration
• Test Strategy & Planning
• Team Leadership

Preferred Skills

• Advanced Python/Scala Development
• API Testing
• AWS Glue / Azure Data Factory / GCP Dataflow
• Data Lineage & Metadata Tools
• Monitoring & Observability Tools
• Machine Learning Pipeline Testing
• Terraform / CloudFormation
• Agile & Scrum Methodologies