Sr. Engineer II, ML Ops
Alegeus · Bengaluru · onsite · Posted 41d ago
Undisclosed7–12 yrsfulltimeSourced from
About a minute. You'll make an account on the way through.
About this role
Role Summary
We are seeking a highly technical Senior/Expert Engineer to establish enterprise-grade AI operations at Alegeus and, once those practices are embedded and team-owned, lead the transition into advanced AI research and emerging capabilities. This is a two-phase role by design: disciplined operational buildout first, then research and innovation leadership not because one matters less than the other, but because research at scale requires the operational foundation to land in production.
The successful candidate is a hands-on builder who can lead. You will own the full AI model lifecycle, govern releases in a regulated environment, train engineers on production practices, and build systems designed from the start to run without your constant involvement. When operations are stable and team-owned, the role pivots deliberately toward emerging architectures, agentic systems, and capabilities that differentiate Alegeus.
Phase 1: Establish Operationalize Own the design and implementation of MLOps and LLMOps practices from scratch. Success here is measured by what the team can do without you.
MLOps LLMOps Pipelines - Design end-to-end model lifecycle: versioning, CI/CD, artifact registry, environment promotion, and deployment automation using Azure Machine Learning, MLflow, and Azure DevOps.
- Build LLMOps practices: prompt versioning, prompt testing harnesses, regression detection across prompt iterations, system instruction governance, and rollback.
- Implement automated retraining with scheduled, drift-triggered, and manual intervention points. Build self-healing mechanisms that detect degradation and remediate without human intervention.
- Build evaluation frameworks for traditional ML and LLM systems: golden datasets, automated test suites, and human-in-the-loop review.
- Monitor model performance, hallucination risk, token cost, latency, and production anomalies. Create dashboards, alerts, and incident response playbooks.
- Establish baseline metrics and production readiness criteria. Own incident triage, root cause analysis, and remediation.
- Define and operationalize an AI Definition of Done: what evidence is required before production release test results, evaluation scores, security review, rollback plan, monitoring readiness.
- Capture operational evidence for all releases: model version, prompt version, approver, deployment environment, evaluation metrics, and production readiness status.
- Implement separation-of-duties controls, human-in-the-loop approval gates, and audit trails for sensitive deployments in healthcare-regulated environments.
- Work with Security, Governance, Risk, Privacy, and Internal Audit to embed compliance into the release lifecycle not bolt it on.
- Build safe rollout patterns: feature flags, canary releases, A/B testing, and phase-gated promotion based on evidence, not intent.
- Establish playbooks, runbooks, and decision frameworks so teams own operations, not just execute tasks.
- Mentor AI engineers, data scientists, and platform engineers on production practices, deployment governance, and incident response.
- Lead architecture reviews and production readiness assessments. Codify patterns. Influence Security, Governance, Product, and Platform stakeholders without direct authority.
Phase 1 is complete when:
Phase 2: Research Innovation Leadership all AI releases follow a shared governed process. Teams operate independently. Audit trails are complete and defensible. You are no longer in the critical path of every deployment.
Operations are team-owned. Your focus shifts to emerging capabilities, applied research, and defining where AI takes Alegeus next. You remain grounded in production constraints research that cannot ship is not research.
Emerging AI Research Assessment - Continuously monitor AI research: large language models, multimodal systems, agentic workflows, advanced RAG architectures, and new model paradigms.
- Evaluate build-versus-buy decisions. Run proof-of-concepts. Convert successful experiments into production-ready capabilities that integrate with the established platform.
- Design autonomous and semi-autonomous AI workflows using agentic frameworks (AutoGen, CrewAI, LangChain, Semantic Kernel). Build orchestration patterns for multi-agent systems.
- Architect advanced RAG systems integrating Alegeus's Snowflake-native data platform with vector databases and embedding models. Improve retrieval quality, grounding, and hallucination controls.
- Extend the AI platform for new paradigms: fine-tuning, distillation, hybrid retrieval, and domain-specific model architectures. Optimize inference cost, latency, and accuracy.
- Drive research into domain-specific problems claims accuracy, document processing, workflow automation and translate findings into product capabilities.
- Shape AI platform strategy. Influence product roadmap through research insights. Partner with engineering, data science, and product leadership on differentiation through AI.
- Mentor next-generation AI practitioners. Share research findings with teams and leadership. Remain hands-on in high-impact proof-of-concepts.
Phase 2 is successful when:
Qualifications 2-3 new AI capabilities have launched from research leadership. The platform supports novel architectures. Product strategy is shaped by your findings. Teams view you as Alegeus's technical North Star for AI.
Required
- 7+ years software engineering; 5+ years leading AI/ML initiatives at production scale
- Deep hands-on expertise in 4+ of: MLOps, LLMOps, evaluation frameworks, drift monitoring, CI/CD for ML, model governance
- Strong Python; able to write, review, and debug production systems
- Experience deploying AI in regulated environments: healthcare, fintech, benefits administration, or equivalent
- Proven ability to establish practices teams adopt and operate independently
- Comfortable with ambiguity; prioritizes ruthlessly between operational rigor and research exploration
Preferred
- Azure ML, MLflow, Azure DevOps, Azure OpenAI, or comparable LLM platforms
- Agentic AI frameworks: AutoGen, CrewAI, LangChain, Semantic Kernel
- Graph DB, Vector databases, embedding models, RAG architectures, context engineering, harness engineering
- Healthcare benefits, claims processing, or consumer-directed healthcare
- Docker, Kubernetes/AKS, Terraform, secrets management, RBAC
- Track record converting AI research into production capabilities
Builder-first: 60-70% hands-on in Phase 1. Writes code, designs systems, ships infrastructure.
Pragmatist: Establishes "good enough" governance first, iterates to enterprise-grade. Balances perfection with progress.
Operations-minded: Thinks in runbooks, monitoring, and audit trails. Owns compliance without being paralyzed by it.
Teacher multiplier: Measures success by what teams can do without them. Codifies patterns, grows others.
Research-curious: Follows AI research. Runs experiments. Comfortable moving between operations and emerging architectures.
Strategist: Sees beyond current products. Articulates a point of view on where AI should take Alegeus.
Sourced from naukri
Skills
AutomationAzure Devopsbenefits administrationAzure Machine Learningai servicesai modelai frameworksAksazure mlAzure OpenAIAi ResearchArchitectureAdministrationAi PlatformMicrosoft AzureClaims ProcessingArtificial Intelligence
About Alegeus
Alegeus
FinTech
501-1000
employees
1995
31 years old
Waltham
US
About
Alegeus Technology
Alegeus is a FinTech company operating in the healthcare benefits space. At a high level, were the leading platform that powers tax-advantaged benefit accounts so think of things like HSAs, FSAs, etc. In other words, we sit at the intersection of healthcare, payments, and technology, and essential we're helping millions of people save smarter and spend better on their healthcare.
In recent years, Alegeus has grown quickly and successfully, but like many fast-growing companies, we’ve reached that stage where we need to evolve from “scale-first strategy” to sustainable, scalable growth. We’re sharpening our identity as a true FinTech company, not just HealthTech and we’re building the operational and strategic infrastructure to support that future.
Since joining Vista Equity Partners, one of the world’s leading private equity firms focused exclusively on software and technology with approx. 80 companies in their portfolio, we’ve been laying the groundwork for that next phase of growth and maturity. We’re investing in technology (think product, AI, etc.) , leadership, and process, all with the goal of creating a more aligned, efficient, and high-performing organization. When it comes to the numbers here's a rapid fire breakdown:
- 1k employees, HQ in Boston, but also have office in Bangalore and Orlando and a presence in Milwaukee.
- We've doubled revenue in five years.
- Directionally we're $300m + this year
- We've tripled EBITDA in five years with 47% EBITDA margins
- We're projected to have more than 11M lives on platform this year, with ~99% client retention
- Greater than 50% rule of 40
Industries
FinTech
Employee ratings
44 reviews
2.5/ 5
Culture
2.5
Career growth
2.2
Work-life
2.9
Employees rate it well for
Senior Management5
Find them on