QA Engineer

As a QA Engineer, you would own quality across Lighthouse and the production AI systems built on it, from requirements through deployment and ongoing operations. This is not a traditional manual testing role; the focus is on test automation, exploratory testing, evaluation design, and ensuring AI/agentic systems are reliable, measurable, secure, and resilient.

The role involves testing APIs, data and context pipelines, agent and workflow orchestration, enterprise integrations and deployment paths, while also validating AI behaviour, including tool use, structured outputs, retrieval/context, state, memory, guardrails, retries and failure recovery.

Requirements:

• 4+ years in QA, test automation, or quality engineering for production software, with strong ownership of backend/API, integration, or enterprise systems.
• Strong hands-on automation skills in Python, JavaScript/TypeScript, or a comparable language, including API testing, SQL/data validation, test design, and CI/CD integration.
• Experience testing distributed/asynchronous systems involving queues, webhooks, background jobs, retries, idempotency, state transitions, and failure recovery.
• Hands-on experience testing LLM, AI, or agentic systems, including evaluation, grounding, tool use, structured outputs, prompt/model regression, safety, and guardrails.
• Ability to translate ambiguous business workflows into risk-based test scenarios, acceptance criteria, edge cases, and meaningful coverage.
• Strong debugging and collaboration skills, with the ability to read logs/traces, reproduce failures, and drive root-cause analysis.

Good to Have:

• Experience with pytest, Playwright/Cypress, Postman/Newman, contract testing, performance testing, and modern test reporting/observability tools.
• Experience with LLM evaluation/red-teaming tools such as Promptfoo, DeepEval, Ragas, or LangSmith, along with RAG/vector search and MCP/tool-server testing.
• Experience with security, permissions, multi-tenancy, private networks, and regulated or customer-hosted deployments.
• This is an opportunity to help define how enterprise AI and agentic systems should be tested in production, with real ownership across quality, automation, AI evaluation, and release confidence.
• If this sounds relevant to your experience, I'd be happy to share the opportunity in more detail over a quick call.