Software Engineer

We are looking an experienced AI Test Engineer to drive quality assurance for AI-powered applications, including LLM models, conversational bots, autonomous agents, and MCP-enabled systems. The ideal candidate will have strong expertise in Playwright with .NET, API/UI automation, AI/LLM validation, and CI/CD automation. The role involves building scalable test frameworks, validating AI responses, testing agent workflows, and ensuring the reliability, security, and performance of GenAI solutions.

Role :AI Test Engineer

Location: PAN India

Experience Range: 5 to 12years

Key Responsibilities:

• Design and execute test strategies for LLM-powered applications, AI chatbots, conversational AI, and autonomous agents.
• Develop and maintain Playwright (.NET) automation frameworks for UI and API testing.
• Validate AI model outputs for:
• Accuracy
• Relevance
• Hallucinations
• Consistency
• Safety and Guardrails
• Test MCP (Model Context Protocol) endpoints and MCP-enabled agent workflows.
• Automate API testing and integrate test suites into CI/CD pipelines.
• Create test datasets, evaluation metrics, and AI validation frameworks.
• Perform regression, functional, integration, and end-to-end testing for AI systems.
• Collaborate with developers, data scientists, and product teams to ensure quality delivery.
• Monitor and troubleshoot AI system failures, agent interactions, and model behavior.
• Implement quality gates within Azure DevOps/GitHub Actions/Jenkins pipelines.