Software Engineer - Gen AI

What you will be doing (Role Accountabilities)

• GenAI Platform Engineering:
• Design, develop, and deploy GenAI solutions using Azure OpenAI Service and AWS Bedrock, selecting appropriate foundation models (GPT-4o, Claude, Titan, Llama, Mistral) based on use-case requirements.
• Build, configure, and publish enterprise copilots using Microsoft Copilot Studio or equivalent low-code/pro-code frameworks, integrating with M365, Teams, SharePoint, and Dataverse.
• Architect and implement RAG (Retrieval-Augmented Generation) pipelines using vector databases (Azure AI Search, OpenSearch, Pinecone) and structured knowledge bases.
• Develop prompt engineering strategies including few-shot prompting, chain-of-thought, and system instruction design — to optimize LLM output quality and consistency.
• Enterprise System Integration:
• Integrate GenAI capabilities with SAP S/4HANA using OData APIs, SAP Business Accelerator Hub connectors, event-driven architectures (SAP Event Mesh, BTP), and workflow triggers
• Build and maintain secure API layers (REST, GraphQL) connecting AI services to ERP, CRM, and operational data sources
• Collaborate with SAP functional and Basis teams to align AI touchpoints with existing business processes, authorization models, and data governance policies
• Architecture & Collaboration:
• Contribute to GenAI architecture decisions including platform selection, cost optimization strategies, multi-region deployment, and security design.
• Mentor junior engineers and participate in internal knowledge sharing through design reviews, documentation, and community of practice sessions.
• Engage with product managers and business stakeholders to translate functional requirements into technically feasible AI solution designs.
• Evaluation, Monitoring & Responsible AI:
• Design and implement LLM evaluation frameworks covering accuracy, hallucination rates, latency, cost per token, and end-user satisfaction metrics.
• Establish monitoring pipelines for production AI applications using tools such as Azure Monitor, AWS CloudWatch, and LangSmith.
• Apply responsible AI practices including content filtering, guardrails (Azure Content Safety, Bedrock Guardrails), bias detection, and explainability mechanisms.
• Manage model versioning, deprecation cycles, and prompt version control to ensure production stability.

What you’ll need to succeed (Skills & Experience)

• 5+ years of software engineering experience, with at least 2 years focused on Generative AI, LLMs, or applied ML in production environments.
• Hands-on experience with Azure OpenAI Service and AWS Bedrock — including model deployment, API integration, token management, and cost governance.
• Practical experience building copilots or AI agents using Microsoft Copilot Studio, Azure AI Studio, LangChain, LlamaIndex, or equivalent agentic frameworks.
• Strong understanding of RAG architecture, vector embeddings, chunking strategies, and retrieval optimization.
• Proficiency in Python (primary) and at least one secondary language (TypeScript, Java, or Node.js).
• Experience integrating AI services with enterprise systems — ideally SAP S/4HANA — using OData, REST APIs, or event-driven patterns.
• Solid understanding of enterprise security requirements including OAuth 2.0, managed identities, RBAC, secret management, and data residency compliance.
• Familiarity with cloud platforms: Microsoft Azure (preferred), AWS, or GCP for deploying and scaling AI workloads.
• Experience with containerization (Docker, Kubernetes) and CI/CD pipelines for AI application deployments.
• Experience with SAP Joule and SAP AI Core.
• Knowledge of SAP BTP (Business Technology Platform) services, SAP Integration Suite, or SAP AI Core