Staff Software Engineer - AI

Responsibilities:

• Build and run the LLM control plane/gateway: smart routing, rate limits/quotas, failover, and token/cost tracking.
• Ship a unified API and SDKs (REST/gRPC) with normalized schemas, structured outputs, caching, and full observability (traces/logs/metrics).
• Enforce safety and privacy by default: content filtering, prompt/response validation, and PII redaction.
• Enable multimodal, multivendor use of LLMs with automated canarying and versioning.
• Own the agent runtime: tool registry, permissions, function calling, grounding, and retrieval.
• Design orchestration patterns (sequential, planner-executor, streaming) and manage agent state and long-running workflows.
• Enable platform components for training and scoring pipelines for classical ML (e. g., XGBoost/LightGBM/linear/trees) and deep models; standardize experiment tracking and packaging.
• Create components to monitor model and data drift, retraining and tuning models as needed to maintain accuracy and relevance.
• Add human-in-the-loop review and safe actioning before agents touch dealer systems.
• Evolve the domain graph and entity resolution; build reliable data ingestion pipelines.
• Serve real-time context to agents (profiles, inventory, pricing, appointments, service history) with access controls and lineage.
• Power retrieval with hybrid search (graph + vector + keyword) and smart cache/TTL to balance accuracy, latency, and cost.
• Run continuous offline/online evaluations for quality, factuality, bias, and safety for platform sanity.
• Define SLOs for latency (p50/p95), uptime, and cost view capabilities; enable autoscaling and spend controls.
• Maintain a model/agent registry, versioning, approvals, audit trails, and reproducibility; support compliance where needed.
• Provide templates/CLIs, sandboxes, and docs so product teams can build and ship fast; mentor engineers and champion MLOps and AI safety best practices.

Requirements:

• 8+ years building large-scale data/ML or platform systems; strong software engineering fundamentals (abstracted API design, concurrency, distributed systems).
• Production experience with Python plus one of Java/Scala/Go; microservices and API design.
• MLOps at scale: pipelines (Airflow/Kubeflow), tracking/registry (MLflow), CI/CD for models, A/B testing, shadow/canary, and online feature computation (Spark/Flink/Kafka).
• Cloud and containers: AWS (preferred), plus Docker/Kubernetes; performance, reliability, and cost engineering in multitenant SaaS.
• Practical ML knowledge (feature engineering, training, evaluation, and drift detection); experience deploying models that power user-facing workflows.
• Built or operated an LLM gateway/control plane: provider adapters, routing/policies, caching, quota/ratelimit, cost, and token accounting.
• Agentic systems: tool use/function calling, orchestration frameworks, human-in-the-loop, safety/guardrails, and online evaluation/telemetry.
• Graph and retrieval: knowledge graphs (e. g., Neo4j/Neptune/TigerGraph), GraphQL, vector search (e. g., pgvector/Qdrant/Milvus), and hybrid retrieval patterns.

Preferred Mindset:

• Platform-as-product: obsess over developer experience, paved roads, and clear SLAs.
• Thinks in systems; observability, fallback, and access control are core, not afterthoughts.
• Passionate about AI; enjoys enabling real-world LLM and agentic use cases.
• Cost-aware builder: you treat latency and dollars as first-class metrics and design for graceful degradation.
• Vendor-agnostic thinker: choose the right model/provider per use case; build for portability and resilience.
• Documentation and teaching: you make complex systems understandable; you uplevel teams.