Senior AI Engineer

Senior AI Engineer — omniXM (AI Platform)

Location: Pune - On-Site

Type: Full-Time

Team: AI Engineering

Reporting To: Head of Engineering

ABOUT OMNIXM omniXM builds intelligent customer experience management products. Our flagship AI product

omniXMOS , includes a conversational survey analysis engine

omniSense , that processes real-world business feedback at scale. The system combines deterministic data tools, LLM-powered reasoning, multi-modal report generation, and document-grounded RAG — all served through a FastAPI backend and deployed on Azure. We are growing our AI Engineering team to deepen the intelligence layer of omniSense: smarter agents, richer retrieval, better-adapted models, and tighter orchestration.

ROLES & RESPONSIBILITIES 1. Agentic AI Design & Orchestration · Design and implement

multi-agent pipelines for survey analysis, insight generation, and automated reporting using frameworks such as LangGraph, CrewAI, or custom orchestration built on FastAPI. · Build

specialized sub-agents (e.g., a data-fetch agent, a metrics-aggregation agent, a narrative-synthesis agent, a report-formatting agent) that coordinate under a planner/orchestrator pattern. · Implement

tool calling that bridges LLM reasoning with deterministic backend tools — including ticket summaries, dimension breakdowns, order/menu extraction, and period comparisons. · Add

Human-in-the-Loop (HITL) checkpoints and agent memory (session context, cross-turn state) to support long-running, multi-step analysis workflows. · Maintain

fault-tolerant execution : retry logic, graceful degradation, and async background task handling for slow or multi-step agentic flows.

2. Retrieval-Augmented Generation (RAG) · Own and extend the document RAG pipelines, including chunking strategies, embedding model selection, vector store management (currently SQLite-backed, with a path to Milvus/Qdrant at scale), and retrieval tuning. · Design

hybrid retrieval combining dense vector search with keyword/BM25 re-ranking to improve precision on business-domain queries. · Build

context injection pipelines that blend retrieved document chunks with live survey response data and precomputed metrics into the PromptBuilder stage. · Evaluate retrieval quality systematically: precision@k, recall@k, faithfulness, and answer relevance against a curated benchmark question set.

3. Fine-Tuning & Model Adaptation · Fine-tune open-source LLMs (LLaMA 3/4, Mistral, Gemma) on omniSense-specific tasks: survey response classification, sentiment labeling, CSAT narrative generation, and structured report drafting. · Apply

LoRA / QLoRA for parameter-efficient adaptation on limited GPU budgets; evaluate Full FT selectively for core classification tasks. · Manage training pipelines: dataset curation, formatting (JSONL instruction format), base model selection, GPU-backed training jobs, eval harness, and versioned model registry. · Deploy fine-tuned adapters behind the existing LLMService abstraction with an inference API (vLLM / llama.cpp / Ollama), measuring latency and accuracy trade-offs vs. frontier models.

4. LLM Prompt Engineering & Evaluation · Maintain and evolve the PromptBuilder staged prompt system — system instructions, context injection, rule sets, few-shot examples, and output format constraints. · Build an

offline eval harness : golden Q&A pairs, regression tests on known failure cases, and automated scoring (LLM-as-judge, exact match, ROUGE/BERTScore). · Run structured A/B experiments across prompt variants, model versions, and retrieval strategies; present results and drive decisions with data.

5. Backend Integration & API Development · Extend FastAPI routers with new AI endpoints: streaming agent responses, async long-running jobs, and structured JSON outputs for frontend consumption. · Integrate with upstream data sources (OmniServices API, Fact API, Ticket API) and ensure the AI layer handles partial failures, stale data, and schema changes gracefully. · Instrument LLM calls with structured logging, token budgeting, cost tracking, and latency metrics (Prometheus / Azure Monitor).

6. Collaboration & Code Quality · Participate in architecture reviews, propose AI capability roadmap items, and translate business requirements into well-scoped AI engineering tasks. · Write unit and integration tests for agent pipelines, RAG retrieval, and LLM tool calls; maintain coverage in tests/unit/ . · Document design decisions, prompt libraries, and model versioning choices so teammates can review and iterate without full context re-loading.

QUALIFICATIONS Required ·

5+ years of hands-on experience building and shipping AI/LLM-powered systems in production. · Proficiency in

Python and async web frameworks (

FastAPI or equivalent). · Practical experience with

multi-agent frameworks : LangGraph, LangChain, CrewAI, Google ADK or equivalent; understanding of ReAct reasoning loops, tool calling, and agent state management. · Hands-on

RAG implementation experience: embedding models (OpenAI, sentence-transformers), vector stores (FAISS, ChromaDB, Qdrant, Milvus), chunking, retrieval tuning. · Familiarity with

LLM fine-tuning : LoRA / QLoRA via Hugging Face PEFT, dataset preparation, training on GPU infrastructure. · Experience working with

OpenAI / Gemini / Anthropic APIs and open-source models (LLaMA, Mistral). · Solid understanding of

prompt engineering : chain-of-thought, few-shot, structured output, function/tool calling schemas. · Experience with

streaming APIs (SSE / WebSocket) for real-time LLM output delivery. · Familiarity with

Azure or other cloud platforms for deployment and storage (Blob, App Service, AKS).

Preferred · Experience with

inference optimization : vLLM, SGLang, Triton Inference Server, llama.cpp, tensor parallelism. · Exposure to

MLOps practices : experiment tracking (MLflow, W&B), model versioning, CI/CD for ML pipelines. · Knowledge of

vector database production operations : indexing strategies, filtering, multi-tenancy. · Experience with

Kubernetes for GPU workload scheduling and scalable LLM serving. · Familiarity with

LLM observability : LangSmith, Helicone, or custom structured logging with token/cost attribution. · Background in

survey analytics, CSAT, CX domains — a strong plus given our product focus. · Contributions to open-source AI projects or published technical writing.

Education · Bachelor's degree or higher in Computer Science, Information Technology, or a related field —

or equivalent demonstrated experience.