Senior QA AI Engineer

Responsibilities

• Lead quality engineering and artificial intelligence testing initiatives to ensure reliable, scalable, and secure software solutions across cloud-based platforms.
• Design, develop, and maintain automated test suites for Java-based backend services and RESTful application programming interfaces to ensure high software quality across releases.
• Own the test strategy for Anthropic Claude integrated features, including large language model output validation, regression baselines, tool-call accuracy verification, context management validation, and end-to-end integration coverage.
• Build and maintain continuous integration integrated evaluation frameworks for non-deterministic artificial intelligence outputs, including rubric-based scoring, automated prompt regression suites, and behavioral drift detection.
• Collaborate with product owners, software engineers, and machine learning engineers to define testable requirements, review acceptance criteria, and ensure high-quality delivery across agile releases.
• Perform database validation testing against PostgreSQL and Microsoft SQL Server environments to verify data integrity, transformation logic, and backend workflow accuracy.
• Conduct artificial intelligence specific security testing, including prompt injection, adversarial input validation, jailbreak testing, and sensitive data leakage assessments aligned with Moody s governance standards.
• Define and enforce synthetic or masked test data standards for artificial intelligence feature testing to ensure appropriate handling of confidential and regulated information.
• Monitor and report quality, performance, and artificial intelligence observability metrics including token consumption, streaming latency, tool-call success rates, model error rates, and reliability indicators.
• Integrate automated testing into continuous integration and continuous delivery pipelines, ensuring quality gates and deployment controls are effective and actionable.
• Conduct test code reviews, mentor quality engineering team members, perform exploratory testing, and drive continuous improvement of testing practices and processes.
• Support performance and load testing of cloud-native software-as-a-service platforms on Amazon Web Services and Microsoft Azure to validate scalability, availability, and reliability requirements.
• Maintain clear test documentation, traceability matrices, defect analysis reports, and quality metrics to ensure transparency and alignment across stakeholders.

Education

• Bachelor or Master of Science in Computer Science, Mathematics, Physics, or a related field

Skills and Competencies

• 5-7 years of experience in software quality engineering with hands-on expertise testing Java-based backend services and RESTful application programming interfaces
• Proficiency with test automation frameworks such as Selenium, TestNG, JUnit, RestAssured, or Cucumber for user interface and application programming interface testing
• Strong understanding of relational databases including PostgreSQL and Microsoft SQL Server to perform data validation and verify data integrity across test scenarios
• Hands-on experience testing Anthropic Claude integrated features, including large language model output validation, semantic evaluation techniques, hallucination detection, and streaming application programming interface behavior
• Familiarity with Model Context Protocol and agentic artificial intelligence testing patterns, including tool-use accuracy, context management, and deterministic versus non-deterministic output validation
• Experience with artificial intelligence security testing, including prompt injection testing, jailbreak validation, sensitive data leakage checks, and application reliability and performance assessments
• Strong understanding of software design principles and microservices architectures, with the ability to develop effective test strategies and identify system-level edge cases
• Deep expertise in artificial intelligence, with a track record of implementing advanced AI solutions to drive strategic transformation and operational efficiency. Strong experience using AI tools to lead innovation initiatives. Demonstrated leadership in managing AI-related risks, ensuring ethical governance, and fostering a culture of responsible AI adoption across the organization.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.