Principal, Software Engineer

It's fun to work in a company where people truly BELIEVE in what they're doing!

Job Description:  Principal, Software Engineer

Ingram Micro is seeking an experienced Principal, Software Engineer that sets the architecture and prompt-engineering direction for the agent layer in the platform, leads code reviews, and writes a meaningful share of the production code themselves.

Key responsibilities

• Own the end-to-end design of new audit capabilities - from data model and prompt structure through API surface, persistence, and React UI.
• Drive prompt engineering and agent design for the verdict-issuing and insight agents, including output_schema design, temperature/grounding decisions, and tool-use patterns.
• Lead code reviews and enforce platform non-negotiables: deterministic temperatures on verdict-issuing agents, Pydantic schema enforcement, salvage-then-repair retries, mandatory evidence_cited and missing_evidence, and constrained MCP tool-use.
• Partner with the Ingram Micro audit team to translate SOPs into machine-readable control logic and SOP JSON.
• Define and uphold the accuracy / hallucination-control program: schema-validity rate, citation re-validation, golden-set regression harness, auditor agreement and override metrics.
• Make build/buy/integration decisions for new connectors across internal Ingram applications, third-party SaaS tools, and infrastructure systems.
• Pair with engineers, unblock them daily, and personally write code on the hardest paths (agent orchestration, schema repair, performance-critical batch flows).
• Represent the team in design and security reviews.

What you bring:

• 8+ years of professional software engineering experience, with 3+ years in a hands-on technical-lead role.
• Strong Python (3.11+): FastAPI or equivalent async web framework, Pydantic v2, asyncio, SQLAlchemy or similar ORM, Postgres.
• Hands-on experience building production AI agents using Google ADK, LangGraph, LlamaIndex, OpenAI Assistants, or comparable agentic frameworks - including tool-use, structured outputs, and multi-agent orchestration.
• Practical prompt engineering against frontier LLMs (Gemini 2.5/3.x, Claude, GPT-4-class), including output_schema / JSON-mode design and reliable structured-output strategies.
• React + TypeScript at a level sufficient to review and contribute to the audit-review UI.
• Production experience integrating MCP (Model Context Protocol) tool servers or equivalent tool-binding patterns.
• Proven track record running accuracy / regression / evaluation programs for LLM-powered systems (golden sets, schema-validity tracking, hallucination guards).
• Containerized deployments (Docker, Cloud Run / GKE / equivalent), Git-based workflows, code review discipline.

Nice-to-have

• Familiarity with ITGC / SOX access-control audits, or experience building compliance, GRC, IAM, or security-tooling products.
• Experience integrating with change-management, HR, and identity/directory systems via REST APIs or equivalent.