Generative AI Test Engineer - Python/Automation Testing

Job Title: GenAI Test Engineer

Location: Chennai / Pune / Hyderabad / Bengaluru / Coimbatore / Ahmedabad / Mumbai

Experience: 69 Years

Employment Type: Full-time

Role Overview We are looking for an experienced GenAI Test Engineer with a strong background in Python-based automation testing and Generative AI applications. The ideal candidate will be responsible for designing and implementing robust testing frameworks for AI-powered solutions, validating Large Language Models (LLMs), RAG pipelines, and ensuring the quality, reliability, and scalability of GenAI systems. This role requires hands-on expertise in Python automation, AI model evaluation, cloud environments, CI/CD integration, and modern software testing practices.

Key Responsibilities

• Design, develop, and maintain end-to-end test architecture for Generative AI applications.
• Build and execute automated testing frameworks for LLM-based applications, RAG systems, and AI workflows.
• Validate the accuracy, reliability, and performance of GenAI models and prompt engineering frameworks.
• Develop automated test suites using Python for APIs, backend services, databases, and AI components.
• Integrate automated testing into CI/CD pipelines to enable continuous quality assurance.
• Perform API, integration, regression, and performance testing across AI-enabled applications.
• Work closely with developers, data scientists, and product teams to identify quality risks and improve overall system reliability.
• Design scalable testing frameworks for cloud-based AI solutions across AWS, Azure, or GCP.
• Monitor test results, analyze failures, and provide actionable recommendations for quality improvements.

Required Skills & Skills

• 69 years of experience in QA Automation Engineering.
• Minimum 2 years of hands-on experience testing Generative AI, AI/ML, or data-intensive applications.
• Strong hands-on experience with Python Automation Testing (Mandatory).
• Experience testing Large Language Models (LLMs) and Generative AI applications (Mandatory).
• Hands-on experience with Retrieval-Augmented Generation (RAG) systems.
• Experience with AI frameworks such as LangChain and AI evaluation tools.
• Strong experience integrating multiple automation tools beyond Selenium.
• Hands-on experience with SQL and Snowflake databases.
• Experience with API testing and backend automation.
• Strong understanding of Software Testing Life Cycle (STLC).
• Experience integrating automated testing into CI/CD pipelines.
• Exposure to cloud platforms such as AWS, Azure, or Google Cloud Platform (GCP).

Preferred Skills

• Experience in prompt engineering validation and AI model evaluation.
• Knowledge of scalable test architecture for cloud-native applications.
• Familiarity with performance and security testing of AI systems.
• Strong analytical, debugging, and problem-solving skills.
• Excellent communication and collaboration skills.

Why Join Us

• Work on cutting-edge Generative AI and LLM-based applications.
• Build scalable automation frameworks for next-generation AI platforms.
• Collaborate with cross-functional engineering and AI teams on enterprise-grade solutions.
• Opportunity to contribute to innovative cloud-native AI products and technologies (ref:hirist.tech)