Machine Learning Engineer

About Valiance

Valiance is a deeptech AI company building sovereign and mission-critical AI solutions for enterprises, public sector, and government institutions. From predictive maintenance and demand planning to sovereign AI for citizen services, we design systems that thrive in high-stakes environments. Recognized with the NASSCOM AI Game Changers Award and the Aegis Graham Bell Award, and a certified Google Cloud Partner, our 200+ engineers and data scientists are shaping the future of industries and societies through responsible AI.

The Role

We are looking for a senior LLMOps Engineer who has taken LLM inference optimization from idea to production — not just proof of concept. You will own the end-to-end efficiency of our LLM inference infrastructure running on GPUs, driving down cost and latency while maintaining the reliability our enterprise and government clients demand. This is a high-ownership, high-impact role on a team building some of India's most consequential AI systems.

What You Will Do

• Design and operate production-grade LLM inference pipelines on GPU clusters, optimizing for throughput, latency, and cost per token.
• Evaluate and deploy small-to-medium open-source LLMs (e.g., Mistral, Llama, Phi, Gemma) as cost-efficient alternatives to large models without sacrificing output quality.
• Tune and manage vLLM deployments — including continuous batching, paged attention, tensor parallelism, and quantization (GPTQ, AWQ, FP8) — in production environments.
• Build and maintain model-serving APIs with robust observability: latency percentiles, GPU utilization, queue depths, and cost-per-request dashboards.
• Architect Kubernetes-based autoscaling strategies for inference workloads, balancing cold-start penalties against cost at scale.
• Run structured A/B experiments comparing model variants, quantization levels, and batching strategies using production traffic — not synthetic benchmarks.
• Collaborate with applied ML engineers and solution architects to identify latency and cost bottlenecks across the model serving stack.
• Establish and enforce SLOs for inference reliability, and build alerting and runbooks for production incidents.

What We Are Looking For

Non-Negotiables

• 3+ years of hands-on experience operating LLM inference in production — demonstrable cost and latency improvements, not POC results.
• Deep expertise with vLLM in production: batching strategies, memory management, quantization tradeoffs.
• Strong Python engineering skills — clean, testable, production-ready code.
• Proficiency with Docker and Kubernetes for deploying and scaling GPU inference workloads.
• Experience building and maintaining REST/gRPC APIs for model serving at scale.
• Hands-on experience with open-source LLMs and the ability to evaluate model-quality vs. cost tradeoffs for real use cases.

Strong Advantages

• Experience with GPU memory profiling and optimization (CUDA-level awareness a plus).
• Familiarity with model distillation, speculative decoding, or flash attention implementations.
• Exposure to multi-GPU and multi-node inference setups.
• Experience with inference frameworks beyond vLLM: TGI, TensorRT-LLM, Triton Inference Server.
• Familiarity with sovereign AI or air-gapped deployment constraints.

Why Valiance

• You will work on AI systems that are actually deployed at scale — used by government institutions and large enterprises, not just demoed.
• Direct access to GPU infrastructure with meaningful compute budgets — no GPU rationing.
• A culture that rewards engineering depth and production ownership over slide decks.
• Competitive compensation with performance-linked incentives.
• Opportunity to define how Valiance builds its AI platform as we scale.

How to Apply

Upload your resume and a brief note on a specific inference optimization you shipped in production — the problem, your approach, and the measurable outcome. We do not conduct screening rounds for this role. Shortlisted candidates will move directly to a technical discussion with our engineering leadership.