RLHF Quality Analyst

About the role

You will judge what a language model produces - ranking responses, applying safety policy and writing the preference data that model training runs on. The work is careful reading and consistent judgement, at volume, against a rubric that keeps moving as the policy sharpens.

• Review and rank model responses for helpfulness, accuracy and safety against a written rubric.
• Apply trust safety policy to borderline content and record why a call was made, not only what it was.
• Flag rubric gaps and recurring ambiguity so the guideline gets fixed rather than worked around.
• Take part in calibration sessions and golden-set checks that keep judgement consistent across the team.
• Maintain the quality bar under deadline without drifting from the rubric.

• Strong reading comprehension and written English - this is the core skill of the role.
• Evident care with detail and the patience for repetitive, judgement-heavy work.
• Willingness to have your judgement measured, discussed and corrected in calibration.
• A bachelors degree in any discipline, or equivalent experience. This role is open to candidates early in their career; the training is provided.
• Helpful, not required: content moderation, editorial, teaching, linguistics or translation experience, or a second Indian language.

Full time, on site at the delivery centre listed above. Standard working hours, with shift cover arranged within the team where a client programme runs to a different timezone.

DesiCrew has run high-stakes AI and enterprise operations from small-town and rural India since 2007. More than 2,000 people work across our delivery centres, 68% of them women - roughly twice the industry average - and 49% the sole earner in their family. We are ISO 9001, ISO 27001 and ISO 42001 certified and a Great Place to Work.

We hire for aptitude and train for the rest. DTOUCH, our in-house learning platform, has delivered more than 5,000 certifications and 1,200 AI-certified professionals, and training continues well past onboarding.