Staff Inbound Product Manager - AI Specialist Evals

Job Summary

AI Specialists don t get better by accident. Hill climbing is how we improve them. We test against real customer environments, find what s failing, fix it, and test again. Now we want every AI Specialist team to climb this way, faster, and without needing us to run it for them. You ll own the product and the program that make that happen.

Responsibilities

• Own the hill climbing product roadmap and scale it across our priority AI Specialists and beyond.
• Run the customer instance program with our business units and product teams, expanding access to real customer environments so every specialist is tested where it actually runs.
• Cut the loop time from a shipped change to a clear, actionable quality verdict from weeks to days.
• Convert eval findings into prioritized fixes with specialist product teams, and stay with those teams until the fixes ship.
• Turn hill climbing into a method product teams can run on their own, with the tools, guides and defaults to do it well and without heavy support from our team.

Qualifications

To be successful in this role you have:

• Experience in leveraging or critically thinking about how to integrate AI into work processes, decision-making, or problem-solving. This may include using AI-powered tools, automating workflows, analyzing AI-driven insights, or exploring AIs potential impact on the function or industry.
• You have 9 or more years of work experience, including product management on AI or ML products in production.
• You ve owned a product roadmap and scaled a product from one team to many.
• You ve run fast improvement loops on ML or LLM systems, from experiments and error analysis to prioritized fixes that shipped.
• You ve taken a process that needed expert support and turned it into a tool or method other teams use without you in the room.
• You can line up product, engineering and business teams behind one plan and keep it moving.
• You have experience with agentic AI, enterprise workflow software or the ServiceNow platform.