Data QA and Automation specialist Databricks, Snowflake & Python)- CTH

Position overview

As an MS-SQL and Databricks Data QA and Automation Specialist, candidate will design automated testing frameworks to validate cloud-based data pipelines, Synapse Datawarehouse, Lakehouse architecture, and Delta Lake migrations. You will ensure extreme data integrity, scalability, and performance across complex AWS data ecosystems.

Essential duties & responsibilities

• Databricks Testing: Build automated validation test suites using PySpark notebooks to test medallion architectures (Bronze, Silver, Gold tables).
• Synapse Datawarehouse Testing: Validate code changes. ETL data movement. Prepare and attach testing plans and scripts
• Automation Engineering: Design and implement robust data-quality frameworks using Python, Great Expectations, or custom PyTest libraries.
• Databricks: Data movement in data bricks in medallion architecture
• Performance Testing: Monitor and analyze Databricks cluster compute efficiency and query performance to optimize processing times and costs.