Quality Analyst

Job description

Role - Quality Analyst - AI/ML (Level 1)
Experience - 1 to 3 years 
Location - Mumbai / Hyderabad (Work from Office )
IMMEDIATE JOINERS ONLY 

Must-have skills - 
Generative AI, LLMs, and Agentic AI Systems
AI Evaluation Frameworks (RAGAS, DeepEval, TruLens, LangSmith, Promptfoo, etc.)
Python and AI/ML Development Libraries
Prompt Engineering, Testing, and Optimization
Automated Evaluation Pipelines and Benchmarking Frameworks
AI Safety, Guardrails, Bias Testing, and Responsible AI Practices
REST APIs, JSON, Vector Databases, and Knowledge Retrieval Systems
Manual Testing of AI/GenAI Applications (Functional, Exploratory, UAT, Regression, E2E)
Agent Reasoning, Tool Calling, Workflow Execution, and Response Validation
Automation Testing using Python Frameworks

Good to Have Skills- 
A/B Testing
Human-in-the-Loop Evaluation
Red Teaming
Evaluation Reporting & Dashboarding
Exposure to RAG-based Applications

Top 3 Responsibilities - 
1. Design, execute, and automate evaluation strategies for Agentic AI applications.
2. Develop evaluation datasets, test cases, and benchmark suites to measure agent performance.
3. Analyze outputs for hallucinations, bias, safety risks, and provide actionable recommendations.

Please apply if you are an immediate joiners