Test Engineer

Role: Senior Test Engineer - AI/ML

Experience: 3-6 years

Location: Mumbai/Bangalore (Hybrid)

Brief About Role:

We are looking for a Senior Test Automation Engineer to join our Quality Engineering (TaaS) team. The ideal candidate should possess strong expertise in modern Quality Engineering practices, UI and API automation, CI/CD integration, and automation framework development. The role requires an engineer who can independently own end-to-end testing activities for enterprise applications while contributing to reusable automation frameworks and AI-powered testing accelerators.

The candidate should be passionate about quality, automation, continuous improvement, and leveraging GenAI to improve testing productivity.

Key Responsibilities:

• Design, develop, and maintain UI and API automation test suites using modern automation frameworks.
• Independently own end-to-end testing activities including test planning, automation development, execution, defect management, and reporting.
• Validate AI/ML applications, Generative AI solutions, LLM-powered applications, and intelligent workflows.
• Design and execute functional, integration, regression, and end-to-end test automation for traditional and AI-enabled applications.
• Develop and maintain reusable automation frameworks, libraries, and utilities.
• Automate API testing using REST Assured (Java) or Python Requests.
• Develop UI automation using Playwright.
• Validate backend data using SQL and perform database verification.
• Develop automated validations for AI model outputs, prompt responses, RAG pipelines, and chatbot conversations.
• Validate AI responses for correctness, consistency, relevance, grounding, safety, hallucinations, and bias.
• Collaborate with ML Engineers, Software Engineers, and Product Owners to define AI quality validation strategies.
• Integrate automated test suites into CI/CD pipelines and implement quality gates.
• Leverage GenAI tools to accelerate test case generation, automation development, debugging, documentation, and productivity.
• Contribute to AI testing frameworks, accelerators, and Quality Engineering best practices.
• Mentor junior engineers and promote automation-first and AI-first Quality Engineering practices.

Must have skills:

• Strong understanding of Software Testing Life Cycle (STLC), SDLC, Agile, and Scrum methodologies.
• Hands-on experience in designing test strategies, test scenarios, test cases, and test execution.
• Strong expertise in UI automation using Playwright and API automation using REST Assured (Java) or Python Requests.
• Hands-on experience with Postman for API testing and validation.
• Strong SQL skills with hands-on experience in database testing and backend data validation.
• Experience integrating automation suites with CI/CD pipelines using Jenkins, GitHub Actions, or similar.
• Proficiency with Git and version control best practices.
• Hands-on experience testing AI/ML or Generative AI applications.
• Understanding of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), chatbots, copilots, or AI agents.
• Experience validating AI responses for accuracy, relevance, consistency, hallucinations, toxicity, bias, and safety.
• Experience designing test scenarios for prompt engineering and AI workflow validation.
• Understanding of AI evaluation techniques and automated output validation.
• Familiarity with AI evaluation frameworks or libraries (e.g., DeepEval, Ragas, LangSmith, Promptfoo, MLflow, or similar).
• Experience using GenAI tools such as ChatGPT, GitHub Copilot, Kiro, Claude, or similar to accelerate test design, automation development, debugging, test data generation, and documentation.
• Strong debugging, root cause analysis, and analytical problem-solving skills.
• Ability to independently own end-to-end testing activities with minimal supervision.
• Excellent communication, collaboration, and stakeholder management skills.
• Ability to mentor junior engineers and promote automation and Quality Engineering best practices.
• Strong attention to detail with a quality-first and automation-first mindset.

Nice to have skills

• Knowledge of AWS services such as Amazon Bedrock, SageMaker, EC2, Lambda, API Gateway, S3, RDS, DynamoDB, CloudWatch, ECS/EKS, and IAM.
• Experience testing cloud-native applications deployed on AWS.
• Experience in Data Warehouse, ETL, or Data Pipeline testing.
• Experience with performance testing tools such as JMeter or k6.
• Exposure to Docker, Kubernetes, and containerized application testing.
• Relevant certifications in Playwright, AWS, Quality Engineering, or Agile are an added advantage.