REMOTEFULLTIME
Sr QA Engineer
Embrace Software
Remote · remote · Posted 3d ago
Sourced from
Undisclosed5–10 yrsfulltimeRemote
Your match
?
Sign in to see your match score, skill gaps & tailored resume.
Section · 01
About this role
We're looking for a Senior QA Engineer to build EPOWERdoc's next-generation quality engineering practice, AI-native from day one. This is a hands-on builder role: you choose and architect our test automation stack, write much of the code yourself, and use AI coding agents every day to design, generate, maintain, and triage tests.
You also own how we test AI itself. You design the evaluation harnesses and guardrails that show EPOWERdoc's AI-powered capabilities are accurate, safe, and dependable enough for an emergency department. You partner closely with developers, product managers, designers, and clinical content experts.
Responsibilities
- Lead QA strategy: Own the end-to-end quality strategy for EPOWERdoc, defining test plans, risk-based coverage goals, and quality gates for every release.
- Architect the QA tech stack: Evaluate, select, and document the tooling for UI, API, performance, accessibility, and security testing, recording each decision and its trade-offs in architecture decision records.
- Build automation and expand coverage: Design and maintain scalable test frameworks for web applications, APIs, and integrations. Set standards clear enough that engineers and AI agents can extend them reliably. Use these frameworks to replace manual regression with automated coverage of critical clinical workflows such as triage, documentation, orders, and discharge.
- Performance and reliability: Build load, stress, and soak tests that model real emergency department peak volumes, establish performance baselines, and catch regressions before they reach production.
- Own CI/CD quality: Integrate automated tests into CI/CD pipelines so every pull request and build is validated quickly. Use parallel execution and AI-assisted test impact analysis to keep feedback loops short as the suite grows.
- Defect management: Work with engineering teams to identify, document, prioritise, and resolve defects early in the development lifecycle, and drive root-cause analysis.
- Exploratory testing: Lead exploratory testing of new features and complex clinical user flows, combining human judgement with AI-driven exploration where automation alone adds less value.
- Agile partnership: Bring a quality perspective to sprint planning, refinement, and retrospectives, including estimates, acceptance criteria, and release readiness.
- Quality advocacy: Champion a quality-first culture from code reviews to release management. Report on quality metrics and trends such as escaped defects, flakiness, and test cycle time.
- Security and compliance: Ensure testing practices, test data, and the use of AI tools meet HIPAA and healthcare data security standards, with no real PHI in test environments or AI prompts.
AI is part of how this team works every day. In this role you will:
- AI-assisted test design: Use LLMs to turn user stories, acceptance criteria, and clinical specifications into test scenarios, edge cases, and boundary conditions, then review and sharpen them with sound test-design judgement.
- Agentic test authoring: Use AI coding agents and Playwrights MCP server and test agents (planner, generator, and healer) to create and maintain automated tests. Hold generated code to the same review and quality bar as production code.
- Self-healing and failure triage: Apply AI to locator healing, flaky-test detection, and automated failure analysis: clustering failures, summarising likely root causes, and routing defects to the right owner.
- Synthetic test data: Make realistic, PHI-free synthetic patient records and clinical scenarios the default for functional, performance, and AI evaluation testing, using tools such as Synthea or LLM-driven generators.
- Testing AI features: Design evaluation frameworks for AI-powered EPOWERdoc capabilities. These include golden datasets, scoring rubrics, LLM-as-judge grading calibrated against human review, and regression gates that run whenever a model, prompt, or knowledge source changes.
- AI safety and guardrails: Test AI features for hallucination, clinical and coding accuracy, bias, PHI leakage, prompt injection, and graceful failure. Work with clinical experts to define what good enough for the ED means in measurable terms.
- Measure and share impact: Track how AI changes test authoring time and maintenance effort, and share what works with other Embrace engineering teams.
This is our starting point, not a mandate. You own the final decisions and the reasoning behind them.
- UI and end-to-end: Playwright with TypeScript, including visual regression and automated accessibility checks (axe-core).
- API and integration: Playwright API testing, contract testing (for example Pact), and validation of HL7 and FHIR interfaces.
- Performance: Locust or k6 for load, stress, and soak testing.
- Security: OWASP ZAP and dependency scanning in CI, plus the OWASP Top 10 for LLM Applications as the baseline for AI features.
- AI tooling: AI coding agents (such as Claude Code, GitHub Copilot, or Cursor), Playwright MCP and test agents, and LLM evaluation frameworks such as Promptfoo, DeepEval, or Ragas, with tracing through Langfuse or LangSmith.
- CI/CD and reporting: Tests running on every pull request, with Allure or a similar dashboard for trends, flakiness, and release readiness.
- First 30 days: You understand EPOWERdoc's clinical workflows, existing test assets, and release process. You've proposed the target QA stack and AI-native approach as written architecture decision records.
- By 90 days: Playwright smoke and regression suites cover the most critical clinical workflows and run on every pull request. A performance baseline exists for peak load. AI-assisted test authoring is in daily use with measured results.
- By six months: Manual regression effort is a small fraction of today's. Flaky tests are tracked and trending down. An evaluation harness gates every release of AI-powered features.
Disclaimer: This job posting and Location has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
Sourced from naukri
Let the agent run this one for you.
Tailored resume, auto-apply, and referral lookup — in under 2 minutes.
Section · 02
Skills
AutomationAgileclaudeCI/CD PipelinesAutomation Toolsanalytical thinkingAI evaluationapisclusteringcd toolsClaude CodeAuthoringBuild AutomationAccessibility TestingAI ToolsArchitectureAPI TestingadvocacyArtificial Intelligence