Artificial Intelligence Architect

Overall Experience

Own the end-to-end technical architecture for a testing agentic AI platform, including multi-agent orchestration, tool interfaces, safety, observability, and scalable deployment.

Years of experience: 8+ years total; 4+ in AI/ML systems; 2+ with LLMs/agentic systems.

Responsibilities

• Define multi-agent system architecture (planner, executor, memory, tool adapters) and governance
• Design tool-use layer (APIs, search, code, test frameworks) with secure sandboxing, enabling agentic systems to operate via tools and feedback loops
• Establish evaluation and safety architecture (guardrails, red-teaming, policy engines) for non-deterministic behavior with human-in-the-loop review
• Select model strategy (proprietary vs OSS), vector/store strategy, and retrieval patterns
• Drive platform non-functionals: reliability, latency, cost, observability, and data privacy
• Collaborate with PO/AI Lead on roadmap and technical debt; work with product teams to design, test, and deploy AI capabilities

Technical Skills

• LLM/Agent Frameworks: LangChain, LlamaIndex, multi-agent orchestration, tool-calling, function calling
• Retrieval & Memory: Vector databases (FAISS, Milvus, pgvector), embeddings
• Evaluation/LLMOps: Prompt versioning, offline/online evaluations, MLflow, Weights & Biases (W&B), feature stores
• Systems: Distributed services, event-driven architecture, gRPC/REST APIs, Kubernetes, Terraform, observability
• Security & Compliance: Data governance, PII handling, secrets management, guardrails

Domain Skills

• Software testing / QA platforms
• Test orchestration and CI/CD integration
• Agentic behavior patterns
• Test generation and coordination at scale