Operations Engineer

Project Role : Operations Engineer

Project Role Description : Support the operations and/or manage delivery for production systems and services based on operational requirements and service agreement.

Must have skills : Splunk Enterprise Architecture and Design, Event management with AIOPS , Splunk Enterprise Observability ITSI

Good to have skills : NA
Minimum 5 year(s) of experience is required

Educational Qualification : 15 years full time education

Summary:
A Tools Platforms Site Reliability Engineer (SRE) ensures the reliability, availability, performance, and continuous improvement of the infrastructure engineering tooling estate - spanning observability platforms, infrastructure-as-code tooling, CI/CD pipelines, ITSM platforms, internal developer portals, secret management, and AI-augmented operations tooling. The role applies a software engineering discipline to platform operations - building automated remediation, establishing SLIs and SLOs for tooling platforms, reducing toil through systematic automation, and owning reliability outcomes end to end across the four tooling pillars.

At Level 7 / 8, this individual operates at the intersection of platform engineering, SRE practice, and AI operations - not just keeping platforms running but continuously raising their reliability ceiling. A distinctive aspect of this role is ownership of LLMOps reliability - ensuring AI-augmented operations tooling (runbook automation pipelines, agentic ITSM workflows, RAG knowledge bases, and AI alert correlation services) meets production-grade SLOs in a regulated financial services environment.
Observability SRE
- ELK/Splunk
- OpenTelemetry
- SLI/SLO/Error Budget
IaC Automation SRE
- Terraform
- Ansible/Chef
- GitHub Actions/ArgoCD
- HashiCorp Vault
- Policy-as-Code
ITSM DevOps SRE
- ServiceNow
- xmatters
- Backstage IDP
- Jira/Confluence
- CMDB Reliability
AI Ops SRE
- LLMOps Reliability
- Agentic ITSM SRE
- AI Alert Pipeline SRE
- RAG Platform SRE
- Model Observability

Roles Responsibilities:
- Own reliability of observability platforms 'splunk- defining and maintaining SLIs, SLOs, and error budgets for metrics pipelines, alerting systems, and dashboard availability across all infrastructure tiers
- Engineer auto-remediation for common observability failures - scraper restarts, index rollover failures, ingest pipeline blockages - reducing MTTR and eliminating repetitive manual toil
- Implement and govern OpenTelemetry instrumentation standards across the infrastructure estate - ensuring telemetry coverage is comprehensive, consistent, and production-grade
- Drive observability-as-code adoption - dashboards, alert rules, SLO definitions, and recording rules version-controlled and deployed through GitOps pipelines with automated testing
- Perform capacity planning and performance analysis for observability platforms - managing cardinality growth, storage retention, query performance, and ingest throughput at scale
- Lead blameless post-mortems for observability platform failures - producing structured RCA with systemic preventive actions that address root causes rather than symptoms
AI-Augmented Operations SRE
- Own reliability of LLMOps pipelines - monitoring model API health (OpenAI, Anthropic Claude, Google Gemini), prompt execution success rates, token consumption, latency SLOs, and cost anomaly alerting for AI-augmented operations tooling
- Engineer reliability for agentic ITSM workflows - LangChain, LlamaIndex, CrewAI - including agent execution health, tool call success rates, human-in-the-loop handoff reliability, and automated failure recovery
- Build observability for RAG knowledge base platforms - vector database (Pinecone, Weaviate, ChromaDB) availability, retrieval latency SLOs, embedding pipeline health, and index freshness monitoring
- Implement AI alert correlation reliability - ensuring LLM-based alert grouping pipelines maintain accuracy and availability SLOs, with fallback to rule-based alerting during AI platform degradation
- Define and enforce LLMOps governance frameworks - prompt version control, model evaluation pipelines, output quality monitoring, and FSI compliance controls (audit logging, data residency) for AI operations tooling
- Lead blameless post-mortems for AI tooling failures - diagnosing model degradation, hallucination events, pipeline failures, and agent workflow breakdowns with preventive actions that meet FSI audit standards

Professional Technical Skills:
Certifications

Terraform Associate or Professional
Splunk Professional

AWS DevOps Engineer Pro or GCP DevOps Engineer
HashiCorp Vault Associate

Certified Kubernetes Administrator (CKA)
ITIL Foundation or Practitioner
Must-Have Technical Skills

Observability SRE: Splunk- SLI/SLO/error budget engineering, OpenTelemetry, ELK/Splunk pipeline reliability, and observability-as-code practices

IaC Reliability: Terraform - state backend health, drift detection automation, module registry SRE, and policy-as-code pipeline reliability across AWS and GCP

CI/CD SRE: GitHub Actions, ArgoCD - pipeline health monitoring, runner auto-scaling, deployment success rate SLOs, and automated rollback engineering

Vault Reliability: HA cluster monitoring, seal/unseal automation, certificate lifecycle management, and lease renewal automation for secrets infrastructure

LLMOps Reliability: Model API health monitoring (OpenAI, Anthropic, Gemini), prompt execution SLOs, token/cost anomaly alerting, and AI pipeline auto-remediation

RAG Platform SRE: Vector database availability (Pinecone, Weaviate, ChromaDB), retrieval latency SLOs, embedding pipeline health, and index freshness monitoring

Automation Toil Reduction: Python - SRE automation scripting, event-driven remediation, infrastructure SDK integration (boto3, GCP client), and operational workflow engineering

Incident Management: P1/P2 bridge leadership, blameless post-mortems, structured RCA, error budget reviews, and SLA-governed resolution in FSI environments

Performance Capacity: Platform capacity trending, SLO burn rate alerting, cardinality management, and proactive capacity interventions across observability and AI tooling
Preferred / Advantageous
- Experience with chaos engineering or game day exercises for platform tooling resilience - validating failure modes in observability, CI/CD, or AI pipeline infrastructure
- Familiarity with eBPF-based observability (Cilium, Pixie) for deep platform telemetry and service mesh reliability engineering
- Exposure to model serving infrastructure - Triton, vLLM, or similar - for AI/ML platform reliability beyond API-based LLM tooling
- Background in SRE or platform engineering within financial services or other highly regulated industries

- SLOs for all platform pillars - observability, IaC, CI/CD, ITSM, and AI tooling - are consistently met, with error budgets actively managed and reliability improving measurably quarter-on-quarter
- Toil across the tooling estate decreases consistently - manual intervention patterns are replaced by automated, observable workflows and the team''s time shifts toward reliability engineering rather than repetitive operations
- Major platform incidents are managed with clear ownership, rapid mobilisation, blameless RCA outputs, and systemic fixes that prevent recurrence
- LLMOps and AI-augmented operations tooling meets production SLOs - model API failures, agent workflow breakdowns, and RAG pipeline degradation are detected early, remediated automatically where possible, and escalated with full context when not
- Engineering teams across Cloud, Network, Security, Database, and Voice towers rely on platform tooling that is observable, self-healing, and consistently available - the Tools Platforms SRE is the reason it stays that way

The candidate should have minimum 5 years of experience in Splunk Enterprise Architecture and Design.

A 15 years full time education is required.

Qualification 15 years full time education