AI Architect

• AI Feature Blueprinting: Create technical blueprints for AI features, including architecture, data needs, model strategy, evaluation approach, and delivery risks.
• Model Selection: Define patterns for identifying which LLM model should be used for each call, and when to use NLP/NLU instead.
• Harness Architecture: Shape reusable patterns for agent loops, tool/function calling, context construction, memory, structured outputs, retries, and failure handling.
• Prompt Engineering: Set standards for prompt design, versioning, review, experimentation, and regression testing as first-class engineering assets.
• Evaluation: Design eval frameworks, golden datasets, regression suites, error analysis, and LLM-as-judge patterns to measure quality and guide decisions.
• RAG & Knowledge Patterns: Guide document parsing, chunking, embeddings, retrieval, reranking, prompt assembly, and grounded response patterns.
• Mentorship & Enablement: Coach P2/P3 AI engineers, review technical designs, create examples, and help junior engineers present architecture findings clearly.
• Cross-Guidewire Collaboration: Partner with Product, AI CoE, PS, customer teams, and SI partners.

KPIs & Success Metrics

• AI feature blueprints are clear, reusable, and actionable for Strike Teams.
• Model and architecture choices are backed by evals, analysis, and documented trade-offs.
• Prompt, harness, RAG, and evaluation patterns are adopted consistently by P2/P3 engineers.
• Delivered AI features meet project bars for accuracy, groundedness, reliability, latency, and cost.
• Junior and mid-level engineers can implement patterns and explain the results confidently.

Key Skills & Experience

• Education: Bachelor's or Master's degree in Computer Science, Data Science, Statistics, Machine Learning, Engineering, or a related technical field.
• Experience: 8+ years of experience in AI, machine learning, data science, software engineering, or solution architecture.
• AI Architecture: Strong ability to translate ambiguous product or delivery goals into practical AI system designs.
• AI Fundamentals: Deep understanding of LLMs, prompts, embeddings, retrieval, structured outputs, tool use, agentic workflows, and model failure modes.
• NLP/NLU Judgment: Able to recognize when classical NLP/NLU, rules-based approaches, lightweight ML, or structured extraction are better than generative AI.
• Technical Proficiency:
  • Strong Python/Java skills and ability to read, review, and contribute to production-quality code.
  • Experience designing LLM harnesses, including orchestration, tool/function calling, context construction, retries, and fallback behavior.
  • Strong prompt engineering practice, including structured iteration, versioning, evaluation, and regression testing.
  • Working knowledge of RAG patterns, embeddings, retrieval evaluation, and vector search concepts.
  • Comfortable designing evaluation approaches using golden sets, regression suites, error taxonomies, and human review loops.
• Engineering Excellence: Familiar with Git, code review, testing, CI/CD concepts, cloud-based delivery, and production engineering practices.
• Communication & Mentorship: Able to explain complex AI trade-offs clearly and coach less experienced engineers without direct reporting authority.

Preferred Skills & Experience

• Agentic AI Systems: Experience with agentic frameworks or custom agent harnesses in production.
• Evals: Experience building structured eval harnesses, regression suites, LLM-as-judge patterns, or production feedback loops.
• Cloud: Hands-on AWS experience with services such as Bedrock, SageMaker, OpenSearch, or related cloud-native AI/data services.
• Fine-Tuning: Experience evaluating or implementing model fine-tuning, distillation, or domain adaptation.
• Guidewire Knowledge: Familiarity with Guidewire products or the insurance domain is a plus.
• Enterprise AI Delivery: Prior experience working on document-heavy, regulated, insurance, finance, or professional services datasets.