Gen AI Contractor

Job Summary

EarnIns Gen AI Team is on a mission to deliver the best agentic experience for our customers - accurate, fast, and set up for success. At the heart of that mission is Evaluation- Driven Development (EDD): a closed loop of human judgment and calibrated LLM scorers that lets us ship new chatbot capabilities with confidence, measure performance in real time, and tie every change to business outcomes.

To help us execute on this vision, we are hiring a Gen AI Contractor for the Analytics Engineering Team. This position is based in Bengaluru, India, requires two days a week in-office as part of our expanding site location, and is a 6-month contract engagement starting in January 2027.

What You'll Do

• Help build and run evaluations for our LLM-powered chatbot systems, contributing to our goal of making evals a standard practice across all teams at EarnIn
• Support the design and calibration of LLM-as-judge scorers, working closely with subject matter experts (SMEs) to align automated scoring with human ground truth
• Contribute to conversation simulation and replay infrastructure - a core building block of our automated research loop
• Assist with real-time observability tooling to detect chatbot performance degradation before customers feel it
• Help analyze production conversations and surface insights that drive chatbot improvements
• Work alongside ML engineers and analysts to iterate on prompt mutations, model changes, and evaluation pipelines

What We're Looking For

• Currently pursuing a Bachelors or Masters degree (B.E., B.Tech, M.Tech, or equivalent) in Computer Science, Software Engineering, or a related field from an Indian university
• Genuine curiosity about large language models, AI evaluation, and building reliable Gen AI systems
• Familiarity with Python and comfort working with APIs, data pipelines, or scripting
• Basic understanding of NLP concepts or hands-on experience with LLMs (even through personal projects or coursework) is a plus
• Strong understanding of probability theory and stochastic systems is a significant plus
• Strong analytical thinking - you ask "how do we know this works" before shipping
• Clear written and verbal communication; you can translate findings for both technical and non-technical audiences
• Self-directed and thorough - you take ownership, follow through, and dont leave things half-done

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.