Senior Software Engineer

Overview About the Role: We are looking for a Senior AI Engineer to join our AI Center of Excellence team at Epsilon, working at the heart of enterprise-grade Conversational, Agentic, and AI Platforms and applications. This role is for someone who brings a strong software engineering backbone, a data-oriented mindset, and deep hands-on expertise in Generative AI, RAG, and Agentic AI systems and is ready to help shape the future of intelligent, enterprise systems at scale. You will be building and scaling AI-powered applications and agents that serve thousands of associates and clients across Epsilon and Publicis Groupe, integrating with enterprise systems. You will own and operate across the full lifecycle - from ideation, experimentation and prototyping to production hardening, evaluation, and operational governance. Click here to view how Epsilon transforms marketing with 1 View, 1 Vision and 1 Voice.

Responsibilities

• Design, develop, and ship production-grade AI applications - including conversational Assistants, RAG pipelines, and multi-agent systems.
• Architect scalable, secure, and cost-efficient backend services using Python, Node.js, and cloud-native patterns (AWS / Azure/ GCP).
• Build and maintain API services (RESTful, streaming) that integrate AI capabilities with enterprise systems.
• Write clean, testable, well-documented code with CI/CD best practices; champion engineering rigor in an AI-first team.
• Generative AI LLM Systems: Build and optimize LLM-powered features - including prompt engineering, structured output design, tool/function calling, and context management (multi-turn conversations, session handling).
• Design and implement evaluation frameworks (groundedness scoring, regression testing, quality benchmarking) for AI outputs - ensuring trust, accuracy, and continuous improvement.
• Stay hands-on with LLM APIs (Azure OpenAI, AWS Bedrock, Anthropic, open-source models) and make informed decisions on model selection, cost-latency tradeoffs, and fine-tuning vs. prompting strategies.
• Design and build enterprise RAG pipelines - including embedding selection, chunking strategies, metadata enrichment, hybrid retrieval, re-ranking, and citation/traceability.
• Integrate and manage vector databases for scalable knowledge retrieval across heterogeneous enterprise data sources.
• Continuously improve retrieval quality by building golden test sets, measuring relevance, and implementing feedback loops.
• Work with Multimodal retrieval based on unstructured content.
• Design and implement agentic workflows - autonomous and semi-autonomous AI agents that can reason, plan, use tools, and execute multi-step business workflows with human-in-the-loop checkpoints.
• Build multi-agent orchestration frameworks using tools like AWS Bedrock, Agentcore, Cursor and other state-of-the-art open-source frameworks - enabling collaborative agent systems for complex enterprise scenarios.
• Develop reusable tool integrations that agents can invoke autonomously, with proper guardrails and safety controls.

Data Analytics Mindset

• Work with structured and unstructured enterprise data - cleaning, transforming, and preparing data for AI consumption.
• Apply data science fundamentals (EDA, statistical analysis, anomaly detection) to diagnose issues, validate model behavior, and derive actionable insights from AI system telemetry.
• Collaborate with data engineering teams to ensure data pipelines are reliable, timely, and aligned with AI feature needs.

Governance, Safety Ops

• Implement Responsible AI practices - including guardrails for hallucination handling, PII protection, restricted topic filtering, and compliance with enterprise security standards.
• Build and operate LLMOps / MLOps pipelines - model deployment, monitoring, logging, tracing, cost tracking, and lifecycle management.
• Contribute to SoPs, governance documentation, and operational runbooks for AI systems deployed across teams.

What Sets You Apart

• You think like a software engineer first - you care about clean code, testable systems, and operational excellence - and you apply that rigor to AI systems.
• You have a builders mindset - youre not just consuming APIs; youre designing platforms, building reusable components, and thinking about how your work scales to 50+ teams.
• You bring data intuition - you can EDA your way through a problem, validate model behavior with data, and explain tradeoffs with metrics.
• You're curious and forward-looking - you track the evolving landscape of AI agents, evaluation, and orchestration and bring those ideas to the team.
• You thrive in a fast-paced, collaborative environment where you work closely with product, operations, and leadership to deliver measurable impact.

Education

• Bachelors or Masters degree in Computer Science, Artificial Intelligence, Data Science, or a related field (or equivalent practical experience).

Must-Have Skills Experience

• 5-8+ years in software engineering, with at least 2+ years hands-on in Generative AI / LLM-based systems
• Software Engineering: Strong proficiency in Python; experience with backend frameworks (FastAPI, Flask, Express/Node.js); clean API design, version control (Git), testing, and CI/CD
• Generative AI: Hands-on experience with LLM APIs (Azure OpenAI, AWS Bedrock, Anthropic, Google Gemini); prompt engineering, structured outputs, tool/function calling
• RAG: Proven experience building RAG pipelines - embedding models, chunking, retrieval logic, vector database, re-ranking, and grounding
• Agentic AI: Experience designing agent-based architectures - tool use, planning, multi-step workflows; familiarity with AWS Bedrock, Azure AI Foundry, or equivalent frameworks
• Data Fundamentals: Solid understanding of data wrangling, SQL, EDA, and basic ML concepts; ability to work with structured/unstructured data at scale
• DataBricks: Designs, builds, and manages scalable data pipelines and AI solutions within the Databricks Lakehouse Platform
• Cloud: Experience with AWS or Azure - deploying containerized services, serverless functions, and working with cloud AI/ML services
• System Design: Ability to design distributed, scalable AI systems with clear tradeoffs on cost, latency, and reliability

Good-to-Have / Forward-Looking Skills

• Multi-Agent Systems A2A Protocols - experience with agent-to-agent communication patterns, Model Context Protocol (MCP), or similar emerging standards.
• Fine-Tuning Model Adaptation - experience fine-tuning LLMs or adapter-based methods (LoRA, QLoRA) for domain-specific use cases.
• AI Evaluation Benchmarking - experience building evaluation harnesses, automated grading, and regression testing for LLM outputs.
• Microsoft Ecosystem - familiarity with M365 Copilot, Copilot Studio, Bot Framework, Teams integrations, Adaptive Cards.
• Observability Tracing - experience with AI-specific observability for debugging and monitoring AI systems in production.
• NLP Classical ML - deeper grounding in NLP (named entity recognition, text classification, sentiment analysis) and classical ML (scikit-learn, XGBoost).
• Knowledge Graphs Hybrid Search - experience combining graph-based retrieval with vector search for richer contextual grounding.
• Edge / Cost Optimization - techniques for reducing inference cost, including model distillation, quantization, caching, and batching strategies.
• Security Compliance - awareness of data privacy regulations, secure API design, and AI red-teaming / adversarial testing.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.