AI Evaluation Professional

Job Summary

As an Art Domain Expert, you will evaluate and improve Large Language Models by applying your expertise across fine arts, visual arts, architecture, design, art history, museums, and artists. You will design challenging prompts, assess AI-generated responses, identify hallucinations and reasoning issues, and contribute to benchmark datasets and adversarial test cases.

Key Responsibilities

• Create advanced prompts covering fine arts, visual arts, architecture, design, art history, museums, and artists.
• Evaluate AI-generated responses for factual accuracy, reasoning quality, completeness, and nuance.
• Identify hallucinations, logical inconsistencies, outdated information, and edge cases.
• Develop benchmark datasets and adversarial test cases.
• Provide evidence-based feedback with reliable references.
• Collaborate with AI researchers to improve model performance.
• Maintain high annotation quality and documentation.

Day-to-Day Tasks

• Write domain-specific prompts of varying difficulty.
• Compare multiple AI responses and rank them.
• Explain why a response is correct or incorrect using authoritative references.
• Identify ambiguous questions and propose improved prompts.

Work Details

• Location: Kolkata, India
• Employment Type: Contractual
• Work Mode: WFH
• Leave Benefits: This contractual engagement does not include paid medical leave or other paid leave benefits.
• Work Schedule: Full-time, 40 hours per week, with at least 4 hours of PST overlap.
• Start Date: Immediate

Job Requirements

Minimum Qualifications

• Master's degree or higher in a discipline related to Art.
• 3+ years of professional, research, or teaching experience.
• Excellent written English and analytical skills.
• Strong attention to detail.

Preferred Qualifications

• Experience with LLMs, Generative AI, prompt engineering, or AI evaluation.
• Published research, industry recognition, or teaching experience is a plus.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.