Data Quality Engineer (Python, PySpark, Playwright & Cloud AI Tools)

Role & responsibilities

Job Title: Data Quality Engineer (Python, PySpark, Playwright & Cloud AI Tools)

Preferred candidate profile

Job Summary

We are seeking a Data Quality Engineer with strong expertise in Python, PySpark, Playwright, Cloud Platforms, and AI-enabled Quality Engineering tools. The ideal candidate will be responsible for ensuring data accuracy, consistency, reliability, and completeness across enterprise data platforms while building automated validation frameworks and leveraging AI-powered testing solutions.

The role requires hands-on experience in data validation, test automation, ETL testing, cloud data platforms, and modern AI-assisted quality engineering practices.

Key Responsibilities

Design and implement scalable Data Quality and Data Validation Frameworks.

Develop automated testing solutions using Python, PySpark, and Playwright.

Validate large-scale data pipelines, ETL workflows, and cloud-based datasets.

Perform source-to-target data reconciliation and quality checks.

Build automated data profiling, cleansing, and anomaly-detection processes.

Create reusable test automation frameworks for data and application validation.

Leverage AI-enabled testing and quality engineering tools to improve coverage and productivity.

Collaborate with Data Engineers, Data Scientists, Business Analysts, and Product Teams.

Execute functional, regression, integration, and data quality testing.

Monitor and resolve data quality issues across production environments.

Generate quality metrics, dashboards, and reports.

Required Skills

Data Quality & Testing

Data Quality Validation

ETL Testing

Data Reconciliation

Data Profiling

Data Governance

Source-to-Target Validation

Data Integrity Testing

Data Quality Monitoring

Programming

Python

PySpark

SQL

Automation Testing

Playwright

Selenium (Preferred)

API Testing

Test Automation Frameworks

Big Data & Data Engineering

Apache Spark

PySpark

Databricks

Delta Lake

Cloud Platforms

AWS

Azure

GCP

Cloud Data Services

Azure Data Factory (ADF)

Azure Databricks

AWS Glue

AWS Redshift

BigQuery

Snowflake

AI-Enabled Quality Engineering Tools

GitHub Copilot

Microsoft Copilot

ChatGPT Enterprise

AI-based Test Automation Tools

Intelligent Test Generation

AI-assisted Data Validation

CI/CD & DevOps

Azure DevOps

Jenkins

GitHub Actions

Git

GitLab CI/CD

Database Technologies

SQL Server

Oracle

PostgreSQL

Snowflake

MongoDB

Preferred Skills

Data Observability Tools (Monte Carlo, Great Expectations, Soda)

Machine Learning Data Validation

GenAI Testing

Prompt Validation

LLM Response Validation

Data Governance Frameworks

Power BI or Tableau Reporting

Qualifications

Bachelor's Degree in Computer Science, Information Technology, Data Engineering, or related field.

Strong analytical and problem-solving skills.

Excellent communication and stakeholder management abilities.

Experience working in Agile/Scrum environments.

Mandatory Skills

Python

PySpark

Playwright

SQL

Data Quality Testing

ETL Testing

Cloud Platform (AWS/Azure/GCP)

Databricks / Spark

CI/CD Tools

Test Automation

Good to Have

Snowflake

Azure Databricks

AWS Glue

Great Expectations

AI-enabled Testing Tools

GitHub Copilot / Microsoft Copilot

Data Observability Tools