Software Test Automation Engineer

Senior Software Test Automation Engineer

Location: Bengaluru – Hybrid

Experience: 7+ Years

Role Overview

We are looking for a Senior Software Test Automation Engineer to drive automation and QA standards for a SaaS-based AI Platform. This is a hands-on individual contributor role focused on Python/Pytest automation, API & UI testing, backend/platform quality, and CI/CD quality gates, with an opportunity to build expertise in LLM and Agent testing.

Key Responsibilities

• Design and maintain robust Pytest-based automation frameworks for Python backend services and agent logic.
• Build automated tests for REST APIs, contracts, schemas, and service-level behaviour using HTTPX, Postman or equivalent tools.
• Develop Playwright-based UI automation for Angular applications.
• Validate SaaS platform capabilities including multi-tenancy, APIs, authentication, event-driven workflows, and database state.
• Integrate automated testing and quality gates into Azure DevOps CI/CD pipelines.
• Define and promote QA automation standards, framework best practices, and reusable testing approaches across multiple engineering squads.
• Contribute to testing strategies for LLM/AI agents, including non-deterministic outputs, multi-step workflows, tool calls, intermediate states, and model-version regressions.
• Collaborate with engineering teams to identify systemic quality gaps and improve automation coverage and test effectiveness.

Must-Have Skills

• 7+ years of QA/Test Automation experience.
• Strong hands-on Python + Pytest experience, preferably with framework design/architecture.
• Strong Playwright experience; Selenium experience may also be considered.
• Experience in REST API automation/testing using HTTPX, Postman or equivalent.
• Experience testing SaaS, backend services, APIs or microservices.
• Strong understanding of SQL/NoSQL for test data and database validation.
• Experience integrating automation into Azure DevOps CI/CD pipelines.
• Understanding of LLM/Agent testing concepts, including non-deterministic outputs and multi-step agent workflows.

Good to Have

• Langfuse or equivalent LLM observability experience.
• Ragas or other LLM evaluation frameworks.
• Experience with event-driven/distributed systems and multi-tenant platforms.
• Performance/load testing experience.
• Experience defining QA standards or mentoring automation engineers.