Test Manager AI Architect / Manager

Key Responsibilities:

• Architect and own AIdriven QA platforms across enterprise programs, ensuring scalability, observability, and governance
• Build, productize, and deploy AI Bots / AI Agents for testing, analytics, and quality decisioning
• Define and drive the endtoend AI strategy and roadmap for the ETMS practice
• Provide handson technical leadership in AI agents, GenAI architectures, and intelligent automation frameworks
• Design enterprisewide LLM integration strategies across:
• Test automation frameworks
• Internal QA tools
• Managed QA platforms
• enabling intelligent automation, AI observability, and quality governance
• Establish AIdriven quality metrics and evaluation frameworks using ML metrics (precision, recall, F1, confusion matrix)
• Lead evaluation, selection, onboarding, and governance of AI tools and platforms
• Integrate AI solutions into CI/CD pipelines for predictive quality, analytics, and automation
• Drive stakeholder communication and AI narrative building, translating complex AI capabilities into business value
• Define data strategy and governance models for AIenabled QA solutions
• Mentor and grow AI architects, test managers, and technical leaders within the practice

Qualifications and Skills:

• Bachelors or master’s degree in computer science, Engineering, Data Science, or equivalent experience.
• 6+ years of experience in QA automation or software engineering.
• 3+ years of experience working with AIdriven automation or intelligent testing frameworks.
• Minimum 2 years of handson experience in building or testing AIbased tools or platforms.

Tools & Technologies

Must Have Tools

• LangChain + LangSmith / LlamaIndex
• DeepEval / RAGAS
• MLflow / Comet ML
• Scikitlearn metrics
• Azure OpenAI / AWS Bedrock

• AutoGen

Good To Have Tools

• Advanced AutoGen setups
• Predictive analytics tools
• AI observability platforms
• Data governance platforms
• CI/CD AI integration tools
• LucidChart / Miro

Nice To Have Tools

• Custom LLM finetuning
• Opensource LLM hosting
• Domainspecific AI stacks
• Multicloud AI tooling
• Researchgrade ML libraries