Staff Test Engineer/ Senior Staff Test Engineer (AI)
Tally Solutions · Bengaluru · onsite · Posted 42d ago
Undisclosed4–8 yrsfulltimeSourced from
About a minute. You'll make an account on the way through.
About this role
Job Summary
We are looking for a Staff Test Engineer / Senior Staff Test Engineer with strong technical expertise and proven experience in AI/LLM-based projects. The candidate will own quality end-to-end, from requirements and architecture discussions through testing, release, and production validation. The role requires a combination of AI testing expertise, strong programming skills, test strategy, risk management, stakeholder communication, and technical leadership. The candidate should be able to independently drive quality for complex products and influence engineering and product decisions.
Responsibilities - Quality Ownership: Own QA activities across the complete project lifecyclefrom requirements and design to release and production validation.
- Test Strategy Planning: Define test strategy, scope, test coverage, automation approach, environments, test data, and release criteria.
- Testing Execution: Drive functional, API, integration, regression, performance, and NFR testing based on project requirements.
- Risk Management: Proactively identify technical, functional, integration, and release risks, define mitigation plans, and communicate them early.
- Release Readiness: Take ownership of release readiness and provide data-driven quality recommendations.
- Production Support: Analyse production issues, drive root-cause analysis, and implement preventive quality improvements.
- Hands-on experience working with AI/ML/LLM-based products or projects.
- Strong understanding of LLMs, Generative AI, Prompt Engineering, and AI-based workflows.
- Experience defining evaluation strategies for AI systems, including accuracy, consistency, hallucination, robustness, context handling, and response quality.
- Experience validating AI agents, tool/function calling, structured outputs, or RAG-based applications is preferred.
- Ability to identify AI-specific edge cases and define appropriate test data, benchmarks, and quality metrics.
- Exposure to LLM evaluation, model comparison, and AI-assisted testing is an advantage.
- Strong programming skills in Python, JavaScript, or similar, with the ability to build test automation and utilities.
- Strong understanding of API testing, databases/SQL, system architecture, networking, distributed systems, and debugging.
- Experience designing scalable automation frameworks and integrating automated tests into CI/CD pipelines.
- Exposure to performance/scalability testing using tools such as JMeter, Gatling, or k6.
- Good understanding of cloud platforms, containers, logging, monitoring, and deployment architecture.
Sourced from naukri
Skills
Customer ExperienceAutomationCraftCI/CD PipelinesDistributed Systemscloud platformshostingApache JMeterGatlingdebuggingdeploymentAI testingJavascriptcloudArchitectureAPI TestingBusiness OperationsGenerative AiLinuxArtificial Intelligence
About Tally Solutions
Tally Solutions
Software Product
1.0k+
employees
1986
40 years old
Bangalore/Bengaluru, Karnataka
India
About
At Tally, we build technology that simplifies businesses, empower people, and creates lasting impact. With over 2.5 million businesses across 100+ countries trusting us.
Industries
Software Product
Employee ratings
423 reviews
4.0/ 5
Culture
4.1
Career growth
3.4
Work-life
4.0
Employees rate it well for
Senior Management43Salary39Work Life Balance37Work Culture36Career Growth29
Find them on