Senior AI Engineer

Senior AI Engineer — omniXM (AI Platform)

Location: Pune - On-Site

Type: Full-Time

Team: AI Engineering

Reporting To: Head of Engineering

ABOUT OMNIXM

omniXM builds intelligent customer experience management products. Our flagship AI product omniXMOS, includes a conversational survey analysis engine omniSense, that processes real-world business feedback at scale. The system combines deterministic data tools, LLM-powered reasoning, multi-modal report generation, and document-grounded RAG — all served through a FastAPI backend and deployed on Azure.

We are growing our AI Engineering team to deepen the intelligence layer of omniSense: smarter agents, richer retrieval, better-adapted models, and tighter orchestration.

ROLES & RESPONSIBILITIES

1. Agentic AI Design & Orchestration

· Design and implement multi-agent pipelines for survey analysis, insight generation, and automated reporting using frameworks such as LangGraph, CrewAI, or custom orchestration built on FastAPI.

· Build specialized sub-agents (e.g., a data-fetch agent, a metrics-aggregation agent, a narrative-synthesis agent, a report-formatting agent) that coordinate under a planner/orchestrator pattern.

· Implement tool calling that bridges LLM reasoning with deterministic backend tools — including ticket summaries, dimension breakdowns, order/menu extraction, and period comparisons.

· Add Human-in-the-Loop (HITL) checkpoints and agent memory (session context, cross-turn state) to support long-running, multi-step analysis workflows.

· Maintain fault-tolerant execution: retry logic, graceful degradation, and async background task handling for slow or multi-step agentic flows.

2. Retrieval-Augmented Generation (RAG)

· Own and extend the document RAG pipelines, including chunking strategies, embedding model selection, vector store management (currently SQLite-backed, with a path to Milvus/Qdrant at scale), and retrieval tuning.

· Design hybrid retrieval combining dense vector search with keyword/BM25 re-ranking to improve precision on business-domain queries.

· Build context injection pipelines that blend retrieved document chunks with live survey response data and precomputed metrics into the PromptBuilder stage.

· Evaluate retrieval quality systematically: precision@k, recall@k, faithfulness, and answer relevance against a curated benchmark question set.

3. Fine-Tuning & Model Adaptation

· Fine-tune open-source LLMs (LLaMA 3/4, Mistral, Gemma) on omniSense-specific tasks: survey response classification, sentiment labeling, CSAT narrative generation, and structured report drafting.

· Apply LoRA / QLoRA for parameter-efficient adaptation on limited GPU budgets; evaluate Full FT selectively for core classification tasks.

· Manage training pipelines: dataset curation, formatting (JSONL instruction format), base model selection, GPU-backed training jobs, eval harness, and versioned model registry.

· Deploy fine-tuned adapters behind the existing LLMService abstraction with an inference API (vLLM / llama.cpp / Ollama), measuring latency and accuracy trade-offs vs. frontier models.

4. LLM Prompt Engineering & Evaluation

· Maintain and evolve the PromptBuilder staged prompt system — system instructions, context injection, rule sets, few-shot examples, and output format constraints.

· Build an offline eval harness: golden Q&A pairs, regression tests on known failure cases, and automated scoring (LLM-as-judge, exact match, ROUGE/BERTScore).

· Run structured A/B experiments across prompt variants, model versions, and retrieval strategies; present results and drive decisions with data.

5. Backend Integration & API Development

· Extend FastAPI routers with new AI endpoints: streaming agent responses, async long-running jobs, and structured JSON outputs for frontend consumption.

· Integrate with upstream data sources (OmniServices API, Fact API, Ticket API) and ensure the AI layer handles partial failures, stale data, and schema changes gracefully.

· Instrument LLM calls with structured logging, token budgeting, cost tracking, and latency metrics (Prometheus / Azure Monitor).

6. Collaboration & Code Quality

· Participate in architecture reviews, propose AI capability roadmap items, and translate business requirements into well-scoped AI engineering tasks.

· Write unit and integration tests for agent pipelines, RAG retrieval, and LLM tool calls; maintain coverage in tests/unit/.

· Document design decisions, prompt libraries, and model versioning choices so teammates can review and iterate without full context re-loading.

QUALIFICATIONS

Required

· 5+ years of hands-on experience building and shipping AI/LLM-powered systems in production.

· Proficiency in Python and async web frameworks (FastAPI or equivalent).

· Practical experience with multi-agent frameworks: LangGraph, LangChain, CrewAI, Google ADK or equivalent; understanding of ReAct reasoning loops, tool calling, and agent state management.

· Hands-on RAG implementation experience: embedding models (OpenAI, sentence-transformers), vector stores (FAISS, ChromaDB, Qdrant, Milvus), chunking, retrieval tuning.

· Familiarity with LLM fine-tuning: LoRA / QLoRA via Hugging Face PEFT, dataset preparation, training on GPU infrastructure.

· Experience working with OpenAI / Gemini / Anthropic APIs and open-source models (LLaMA, Mistral).

· Solid understanding of prompt engineering: chain-of-thought, few-shot, structured output, function/tool calling schemas.

· Experience with streaming APIs (SSE / WebSocket) for real-time LLM output delivery.

· Familiarity with Azure or other cloud platforms for deployment and storage (Blob, App Service, AKS).

Preferred

· Experience with inference optimization: vLLM, SGLang, Triton Inference Server, llama.cpp, tensor parallelism.

· Exposure to MLOps practices: experiment tracking (MLflow, W&B), model versioning, CI/CD for ML pipelines.

· Knowledge of vector database production operations: indexing strategies, filtering, multi-tenancy.

· Experience with Kubernetes for GPU workload scheduling and scalable LLM serving.

· Familiarity with LLM observability: LangSmith, Helicone, or custom structured logging with token/cost attribution.

· Background in survey analytics, CSAT, CX domains — a strong plus given our product focus.

· Contributions to open-source AI projects or published technical writing.

Education

· Bachelor's degree or higher in Computer Science, Information Technology, or a related field — or equivalent demonstrated experience.