HYDERABAD · FULLTIME
AI QA Engineer Consultant
AceNet
Hyderabad · onsite · Posted 12d ago
Sourced from
Undisclosed5–10 yrsfulltimeHyderabad
Your match
?
Sign in to see your match score, skill gaps & tailored resume.
Section · 01
About this role
Job Summary
Role Overview
Key Responsibilities
Must-Have Skills
Good to Have
Niche Positioning
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
AI QA Engineer / AI QA Consultant
Role Overview
We are seeking an AI QA Engineer / AI QA Consultant responsible for ensuring the quality, reliability, performance, robustness, and compliance of AI/ML, Generative AI, LLM, and Agentic AI solutions. This role operates within pharmaceutical/life sciences and manufacturing contexts, aligning AI testing, governance, and quality practices with regulatory expectations and production-readiness requirements. The ideal candidate blends deep AI model validation expertise with practical quality engineering discipline to enable safe, auditable AI deployments.
Key Responsibilities
- Develop and execute AI/ML model QA and evaluation strategies across the model lifecycle, from development through production.
- Evaluate ML models using metrics such as accuracy, precision, recall, F1 score, ROC/AUC, confusion matrix, and domain-specific thresholds.
- Test models across diverse datasets, edge cases, failure scenarios, and varying operating conditions to ensure robust performance.
- Perform GenAI/LLM evaluation covering accuracy, relevance, consistency, hallucination, groundedness, and response quality.
- Design and execute testing for AI agents and multi-agent workflows, including orchestration, tool calling, and output validation.
- Build test datasets, evaluation frameworks, test harnesses, and automated QA pipelines tailored to AI systems.
- Utilize Python, APIs, scripting, and automation to conduct AI testing and evaluation at scale.
- Work with LangChain, LangGraph, Langfuse, and similar AI observability and evaluation tools to monitor and validate agent behavior.
- Identify bias, model drift, hallucination, instability, and AI application/model failures, followed by root-cause analysis and remediation planning.
- Define test cases, user stories, acceptance criteria, quality thresholds, and production-readiness criteria in collaboration with stakeholders.
- Collaborate with Data Scientists, ML Engineers, Software Engineers, Product Owners, and business stakeholders to align on quality objectives.
- Document test results, model limitations, risks, defects, and evaluation outcomes in audit-ready formats suitable for regulatory review.
- Support Responsible AI, AI Governance, Data Integrity, and Quality-by-Design practices within project governance.
Must-Have Skills
- Machine Learning Model Evaluation
- GenAI / LLM Evaluation
- Agentic AI Multi-Agent Systems
- Python, APIs Automation
- LangChain, LangGraph, Langfuse
- Prompt Engineering LLM Testing
- AI Risk, Bias Robustness Testing
- Model / AI Application Regression Testing
- AI Test Strategy Test Case Design
Good to Have
- Pharmaceutical / Life Sciences experience
- GxP / CSV / Data Integrity knowledge
- Azure / AWS / GCP
- MLOps / LLMOps / ModelOps
- RAG and Vector Databases
- Responsible AI AI Governance
- CI/CD and automated testing
- AI Red-Teaming / Adversarial Testing Experience
- 5+ years of experience in AI/ML, GenAI, Model QA/Validation, Software QA, Data Science, or Quality Engineering.
- Experience in regulated, pharmaceutical, life sciences, manufacturing, or other quality-critical environments is preferred (not mandatory).
Niche Positioning
This role targets professionals specialized in AI Quality Engineering, AI Model Validation, GenAI Testing, LLM Evaluation, and Agentic AI Testing. It differentiates from traditional software QA by emphasizing AI/ML-driven quality assurance, governance, and production readiness. The candidate synthesizes AI/ML expertise with quality engineering disciplines to deliver auditable, trustworthy AI systems in regulated contexts.
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
Sourced from naukri
Let the agent run this one for you.
Tailored resume, auto-apply, and referral lookup — in under 2 minutes.
Section · 02
Skills
AutomationCI/CD Pipelinescloud platformsapisautomated testingai modelamazon web servicesai technologiesCSVai governanceAnalyticsCloud InfrastructureAI testingcloudBusiness TransformationArchitecturecomplianceAi SolutionsMicrosoft AzureArtificial Intelligence
Section · Company
About AceNet
AceNet
Management Consulting
25
employees
2019
7 years old
Gurgaon/Gurugram, Haryana
India
₹13.9L PA avg
Avg at AceNet
About
AceNet provides a global network for current and former executives, facilitating knowledge sharing and decision support across various industries.
Industries
Management Consulting
Employee ratings
7 reviews
4.0/ 5
Culture
3.1
Career growth
3.1
Work-life
3.4
Employees rate it well for
Overall Company40Compensation & Benefits35
Find them on