LLM Scientist

Role Overview
We are looking for an experienced LLM Scientist to research, design, and prototype advanced Generative AI systems. This role is experimentation- and coding-heavy, with a strong focus on Agentic AI design, RAG architecture, LLM fine-tuning, and rapid Python-based prototyping of new GenAI techniques ahead of production handoff.

Key Responsibilities

Agentic AI & GenAI Prototyping

• Design and prototype Agentic AI / Multi-Agent systems using LangGraph, LangChain, and MCP.
• Research and experiment with emerging LLM and Agentic AI techniques.
• Rapidly prototype and validate new GenAI approaches using Python.

• LLM Fine-Tuning & Experimentation
• Fine-tune LLMs using LoRA/QLoRA.
• Conduct experiments comparing models, prompting strategies, and architectures.
• Run controlled experiments, including A/B tests and ablation studies.
• Apply statistical rigor when designing experiments and interpreting results.

• RAG Architecture
• Architect and iterate on RAG pipelines.
• Experiment with chunking strategies, embeddings, hybrid search, and reranking.
• Evaluate retrieval quality and optimize retrieval strategies.

1. Evaluation & Benchmarking

• Build evaluation and benchmarking frameworks for GenAI systems.
• Evaluate retrieval quality, groundedness, hallucination, and relevance.
• Implement observability and tracing using tools such as LangFuse.
• Develop robust evaluation methodologies and avoid metric gaming.

Machine Learning & Statistical Analysis

• Apply statistical concepts including hypothesis testing, confidence intervals, significance testing, and sample-size considerations.
• Design and evaluate supervised, unsupervised, and semi-supervised learning approaches.
• Analyze bias-variance tradeoff, regularization, and overfitting/underfitting.
• Apply classical ML algorithms and evaluation metrics to support GenAI system development.

• Collaboration & Research
• Track emerging LLM and Agentic AI research and assess applicability to business problems.
• Partner with LLM Engineers to hand off validated prototypes for production scaling.
• Research and evaluate new approaches to improve GenAI system performance.

Required Skills

ML & Statistical Foundations

• Strong understanding of statistics and probability, including distributions, hypothesis testing, confidence intervals, and significance testing.
• Strong understanding of supervised, unsupervised, and semi-supervised learning paradigms.
• Knowledge of bias-variance tradeoff, regularization, and overfitting/underfitting diagnosis.
• Experience with model evaluation metrics such as Precision, Recall, F1, and ROC-AUC.
• Practical experience with classical ML algorithms including regression, classification, clustering, and ensemble methods.
• Understanding of optimization fundamentals such as gradient descent, learning-rate tuning, and convergence behavior.

GenAI & Engineering Skills

• Strong Python coding skills with experience building experimental pipelines and tooling from scratch.
• Deep understanding of LLMs, Transformer architecture, and Generative AI fundamentals.
• Hands-on experience with Agentic AI, RAG, and LLM fine-tuning.
• Proficiency in LangChain, LangGraph, and MCP (Model Context Protocol).
• Experience with vector databases such as FAISS, ChromaDB, and Pinecone.
• Strong understanding of GenAI evaluation methodologies, including groundedness, hallucination detection, and relevance scoring.

Good to Have

• Knowledge of NLP techniques such as BERT, embeddings, and sentiment analysis.
• Experience with classical ML techniques such as XGBoost and anomaly detection.
• Research background or publications in NLP/LLMs.
• Experience working in regulated enterprise domains.
• B.Tech/M.Tech in Computer Science, IT, or a related field.