Automation Test Engineer (AI & Agent)

Role: Automation Test Engineer (AI & Agent)

Experience: 6-12 Years

Location: Hyderabad, Bangalore, Chennai, Delhi

Job Description:

Key Responsibilities

• Define risk-based test strategies, quality gates, coverage models, and release criteria for AI agents and automation solutions.
• Design automated testing frameworks for prompts, tools, workflows, memory, grounding, governance controls, integrations, and human approval processes.
• Develop golden datasets, scenario libraries, adversarial test cases, and outcome-evaluation frameworks with business and technical stakeholders.
• Execute functional, regression, integration, end-to-end, performance, resilience, recovery, and operational-readiness testing.
• Evaluate hallucination risks, grounding accuracy, relevance, consistency, safety controls, tool execution quality, and decision traceability.
• Validate access controls, prompt-injection protection, data leakage prevention, auditability, reversibility, and fail-safe mechanisms.
• Test integrations across enterprise systems, operational support platforms, service management systems, inventory repositories, telemetry sources, and workflow orchestration platforms.
• Automate evidence collection within CI/CD pipelines and maintain independent test environments, mocks, stubs, and virtualized services.
• Manage defect identification, prioritization, root-cause validation, and remediation verification.
• Provide release-readiness assessments and independent quality recommendations for testing, deployment, and production releases.

Mandatory Technical Skills

AI & Agent Testing

• LLM evaluation and benchmarking
• Prompt regression testing
• Function and tool validation
• Grounding validation
• Memory testing
• Multi-agent workflow testing

Automation Frameworks

• Python
• PyTest
• API and UI testing
• CI/CD pipelines
• Test data management
• Mocks, stubs, and service virtualization