AI Infrastructure & LLMOps Architect

About KPMG in India

KPMG entities in India are professional services firm(s). These Indian member firms are affiliated with KPMG International Limited. KPMG was established in India in August 1993. Our professionals leverage the global network of firms, and are conversant with local laws, regulations, markets and competition. KPMG has offices across India in Ahmedabad, Bengaluru, Chandigarh, Chennai, Gurugram, Jaipur, Hyderabad, Jaipur, Kochi, Kolkata, Mumbai, Noida, Pune, Vadodara and Vijayawada.

KPMG entities in India offer services to national and international clients in India across sectors. We strive to provide rapid, performance-based, industry-focused and technology-enabled services, which reflect a shared knowledge of global and local industries and our experience of the Indian business environment

AI Infrastructure & LLMOps Architect

Experience: 8–12 years; 2+ years supporting production AI/ML workloads

Locations: Delhi NCR / Bengaluru / Mumbai

Role type: Full-time | Enterprise AI Technology & Engineering

Role purpose

Build and operate secure, reliable and cost-efficient infrastructure for enterprise AI, including model inference,

GPU compute, deployment automation and observability.

Key responsibilities

• Architect cloud, hybrid and private AI platforms: compute, GPUs, storage, networking and containers.

AI Solutions

Enterprise AI Technology & Engineering | Recruitment JDs 2

• Build and tune LLM serving environments; implement autoscaling, batching, caching and workload

scheduling.

• Create CI/CD, infrastructure-as-code, model deployment, monitoring, versioning and rollback pipelines.

• Implement controls for IAM, secrets, network isolation, data residency, disaster recovery and production

SLAs.

• Optimise GPU utilisation, inference throughput, latency and cost; partner with application architects.

Required capabilities

• Deep AWS, Azure or GCP infrastructure experience; Docker, Kubernetes, Helm and Terraform.

• Model serving with vLLM, NVIDIA Triton, TGI or comparable technology.

• Python/Bash, Git-based CI/CD, Linux and platform automation.

• Monitoring with Prometheus, Grafana, OpenTelemetry or comparable tools.

• Working understanding of GPUs, inference performance, security and reliability engineering.

Preferred experience

• NVIDIA NIM / AI Enterprise, Ray, KServe, MLflow or Kubeflow.

• Distributed inference, quantisation, private-cloud deployment and GPU capacity planning.

Measures of success

• Reliable inference platforms with production SLAs.

• Improved latency, throughput and GPU utilisation.

• Lower inference cost and faster, safer deployment cycles.

Equal employment opportunity information

KPMG India has a policy of providing equal opportunity for all applicants and employees regardless of their color, caste, religion, age, sex/gender, national origin, citizenship, sexual orientation, gender identity or expression, disability or other legally protected status. KPMG India values diversity and we request you to submit the details below to support us in our endeavor for diversity. Providing the below information is voluntary and refusal to submit such information will not be prejudicial to you.