AI Agent Quality Analyst
CIS Technologies · Kochi · onsite · Posted 64d ago
Undisclosed1–6 yrsfulltimeSourced from
About a minute. You'll make an account on the way through.
About this role
Were looking for an AI Agent Quality Analyst to monitor and evaluate the performance of our AI agents in production. This role identifies recurring issues and failure patterns, diagnoses likely causes, and works closely with product and engineering to drive improvements. It sits at the intersection of product and engineering, translating technical findings into clear, actionable insight for both sides.
Key Responsibilities
- Continuously review AI agent outputs to identify issues, with a focus on recurring patterns rather than isolated incidents
- Investigate outputs, logs, and correction history to determine where and why agents underperform
- Use Python to extract, organize, and analyze the data needed to support findings
- Communicate issues and patterns clearly to both technical and non technical stakeholders
- Recommend likely root causes and directions for improvement
- Maintain an ongoing record of issues and patterns to track trends over time
- Partner with the product team to understand intended agent behavior and business context
- Work with engineering to ensure findings are well understood and actionable
- Distinguish between data or coverage gaps and genuine model or logic failures
- Strong analytical skills with demonstrated ability to identify patterns in data
- Working proficiency in Python for data extraction and analysis (e.g. pandas, basic scripting against APIs or databases)
- Comfortable working with structured data from a relational database (PostgreSQL or similar)
- Excellent written and verbal communication skills, with the ability to translate technical findings for non technical audiences
- Strong cross functional collaboration skills
- Disciplined and consistent documentation habits
- Understanding of RAG (retrieval augmented generation) concepts, including retrieval and ranking
- Prior experience reviewing AI or LLM generated content
- A clear, current picture of common failure patterns across AI agents
- Faster diagnosis-to-fix cycles for engineering
- Clear visibility for product into data gaps versus model behavior issues
- A reliable historical record of issue trends over time
Who we are:
At CIED Group, we are on a mission to make the global food supply chain more efficient, intelligent and transparent. We design technologies that build data trust and drive productivity across the agri-food supply chain, from the farm to the consumers plate.
Our flagship product, RightOrigins, is an agentic AI platform that drives automation throughout the supply chain workflows. We also provide traceability services and build custom audit management tools for food certification bodies, simplifying regulatory compliance and audit inspection processes with a digital, collaborative approach.
At CIED Group, every idea is built around one core belief: when data drives credibility, trust follows naturally.
What we offer:
- Friendly environment
- Chance to own responsibility and demonstrate accountability
- Chance to develop innovative skills ideas
- International horizons - work with international teams and clients from all corners of the world
- Company parties, team events and the opportunity to work with great colleagues from all over the world
- Medical Insurance
Industry: IT
Employment Type: Full-time
Location: Currently Remote/Hybrid
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
Sourced from naukri
Skills
data extractionAutomationPostgresqlAuditinganalytical thinkingmedical insurancePythonRegulatory ComplianceSupply ChainQuality Analyst
About CIS Technologies
CIS Technologies
Engineering & Construction
Hyderabad / Secunderabad, Telangana
India
₹8.6L PA avg
Avg at CIS Technologies
About
Ashiyana Dream Homes Pvt. Ltd. is a real estate development company committed to creating quality residential and commercial spaces. The company focuses on delivering well-planned, affordable, and sustainable developments with an emphasis on quality construction, customer satisfaction, and timely project execution.
Industries
Engineering & Construction
Employee ratings
10 reviews
4.1/ 5
Culture
4.4
Career growth
4.2
Work-life
4.3
Employees rate it well for
Company Culture42Work-Life Balance41
Find them on