Sr Quality Engineer

Performance Testing Engineer (AI)


What Youll Do

• Design and run performance, load, stress, soak, and scalability tests for applications, APIs, and data pipelines, in close partnership with developers and business SMEs.
• Performance-test AI systems (LLM gateway, shared RAG services, MCP servers, and agents), measuring time to first token, end-to-end latency, throughput, concurrency limits, rate-limit behavior, and token cost per transaction.
• Build AI-assisted test automation: use LLMs and agents to generate scripts, create realistic synthetic workloads and data, triage results, and draft reports.
• Integrate performance tests into CI/CD pipelines with SLO-based thresholds and gates so regressions are caught before release.
• Instrument and monitor systems with observability tooling, diagnose bottlenecks across the application, database, network, and infrastructure layers, and recommend fixes.
• Build evaluation harnesses that track AI output quality alongside performance, so speed and cost optimizations do not quietly degrade answers.
• Run workshops with engineering and business teams on performance testing practices and on using tools like Claude Code in testing workflows.
• Feed benchmarks, patterns, anti-patterns, and platform gaps back into the Center of Excellence so the next engagement moves faster.

Must Have Experience

• Hands-on performance testing of production systems using tools such as JMeter, k6, Gatling, or Locust.
• Production-quality code in at least one of Python, JavaScript/Node, or Java/.NET, enough to build custom load generators and test automation.
• Experience with observability and APM tools (such as Datadog, Dynatrace, New Relic, Grafana, or Azure Monitor) and with analyzing metrics, traces, and logs to find root causes.
• Working knowledge of cloud infrastructure (Azure or AWS) and containers, and of how relational, NoSQL, and vector data stores behave under load.
• Used AI tools or LLM APIs in real engineering or testing workflows, with a grounded understanding of how LLM systems behave under load: token limits, rate limiting, streaming, caching, and concurrency.

Traits

• Strong written and verbal communication; able to explain results and risk to non-engineers.
• Pragmatic judgment on what to test, how deep to go, and when results are good enough to ship.
• Systems thinking with a focus on failure modes; AI helps write the tests, you own the results.
• Comfort running multiple agents and automations in parallel as part of daily workflow.

Location


Scheduled Weekly Hours


Job Type


Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.