Senior AI QA Engineer

Role: Sr. AI QA Engineer

Location: Noida / Gurugram / Pune / Mumbai

Employment Type: Full-Time

About the Role

We are looking for a highly skilled AI QA Engineer to join our Quality Engineering team. This role combines traditional automation testing expertise with modern AI application validation. The ideal candidate will have hands-on experience testing GenAI applications, LLM-powered systems, chatbots, RAG applications, and AI-driven workflows, along with strong automation experience using Playwright.

You will work closely with Product, Engineering, Data Science, and AI teams to ensure the quality, reliability, and accuracy of next-generation AI-powered products.

Key Responsibilities

• Design, develop, and maintain test automation frameworks using Playwright and Selenium.
• Perform end-to-end testing of web applications and AI-powered user experiences.
• Validate chatbot conversations, AI-generated responses, and conversational workflows.
• Execute testing for GenAI, LLM, and RAG (Retrieval-Augmented Generation) applications.
• Perform prompt testing, grounding validation, and hallucination detection.
• Validate AI agent workflows, multi-step reasoning, and tool integrations.
• Execute functional, regression, integration, API, database, and UAT testing.
• Perform SQL-based backend validations and data quality checks.
• Collaborate with cross-functional teams to define and implement quality strategies.
• Participate in Agile ceremonies and contribute to continuous quality improvement.

Must-Have Qualifications

• Bachelor's degree in Engineering (B.E./B.Tech) in Computer Science, Information Technology, Electronics, or related fields.
• Strong hands-on experience with Playwright (Mandatory).
• Experience with Selenium Automation.
• Experience testing GenAI, LLM, Chatbot, or AI-powered applications.
• Hands-on experience with RAG Testing.
• Strong understanding of:
• Prompt Testing
• Hallucination Testing
• Response Validation
• Grounding Validation
• Experience with API Testing using Postman or REST APIs.
• Strong SQL skills and experience with databases such as:
• MySQL
• PostgreSQL
• MongoDB
• DynamoDB
• Experience with Python or Java.
• Familiarity with:
• Git
• Jira
• CI/CD Pipelines
• Agile Methodologies

Good-to-Have

• Experience with Agentic AI Testing
• Experience with MCP (Model Context Protocol)
• Exposure to LangSmith, Ragas, Promptfoo, DeepEval
• Experience with vector databases:
• Pinecone
• ChromaDB
• Weaviate
• FAISS
• AWS or GCP experience
• Exposure to Amazon Bedrock or Azure OpenAI

Preferred Project Experience

Candidates with experience in any of the following will be highly preferred:

GenAI & Chatbot Testing

• Conversational AI testing
• Prompt validation
• LLM response evaluation
• Hallucination detection

RAG Application Testing

• Retrieval quality validation
• Embedding and chunking validation
• Knowledge grounding verification
• Citation accuracy testing

Agentic AI Testing

• Multi-agent workflows
• Autonomous decision-making systems
• AI workflow orchestration
• Tool and API integrations

Skills

Playwright, Selenium, Generative AI, LLM Testing, RAG Testing, AI Testing, Chatbot Testing, Prompt Testing, Agentic AI, Python, Java, API Testing, Postman, SQL, AWS, CI/CD, Jira, Test Automation, Quality Assurance

Why Join Us

Join a team building innovative AI-powered products where you'll play a critical role in ensuring reliability, accuracy, and exceptional user experiences through advanced AI quality engineering practices.