AI Test Manager

Test Manager – AI / GenAI Quality Assurance

Role Overview

We are seeking an experienced and detail-oriented Test Manager to manage end-to-end quality assurance activities for Artificial Intelligence (AI), Generative AI (GenAI), and Agentic AI solutions. The ideal candidate will have strong experience in QA delivery management, AI testing practices, automation frameworks, and enterprise testing methodologies.

The role will focus on managing testing activities, coordinating with cross-functional teams, ensuring quality governance, and supporting successful delivery of AI-enabled applications within Agile and DevSecOps environments.

Key Responsibilities

Test Management & Delivery

• Manage end-to-end testing activities across AI, GenAI, and Agentic AI projects.
• Prepare and maintain test strategies, test plans, test schedules, and QA deliverables.
• Coordinate testing activities across functional, integration, regression, performance, compatibility, and UAT testing streams.
• Monitor testing progress, risks, defects, and quality metrics to ensure timely delivery.
• Work closely with project managers, developers, business analysts, product owners, and DevOps teams to ensure quality objectives are achieved.
• Support release management, deployment validation, and production readiness activities.

AI / GenAI / Agentic AI Testing

• Manage testing and validation of AI/ML applications and Generative AI solutions.
• Support validation of:
• AI model outputs
• Prompt engineering effectiveness
• Agentic AI behavior and interaction patterns
• Goal achievement and response accuracy
• Fairness, bias detection, explainability, privacy, and security
• Ensure Responsible AI testing standards and governance practices are followed.
• Coordinate testing for AI performance, reliability, scalability, and compliance requirements.

Automation & DevSecOps Collaboration

• Oversee automation testing activities using tools such as:
• Selenium
• Tosca
• Python-based frameworks
• Collaborate with DevSecOps teams to integrate testing within CI/CD pipelines.
• Ensure automation coverage and quality gates are maintained across releases.
• Support defect triage, root cause analysis, and issue resolution processes.

Stakeholder Coordination & Reporting

• Provide regular QA status updates, metrics, and risk reports to stakeholders.
• Facilitate defect review meetings, UAT coordination, and testing sign-offs.
• Maintain accurate tracking and reporting within tools such as Jira and Azure DevOps.
• Ensure effective communication between QA, development, and business teams.

Required Skills & Experience

• Proven experience as a Test Manager or QA Manager with exposure to AI/ML testing environments.
• Strong experience in testing AI applications, including GenAI and Agentic AI solutions.
• Knowledge of Responsible AI principles including:
• Fairness
• Bias detection
• Explainability
• Privacy
• Security
• Strong understanding of QA methodologies including Agile, Waterfall, DevOps, and BDD.
• Hands-on experience with automation testing tools such as Selenium, Tosca, and Python.
• Experience working within DevSecOps and CI/CD environments.
• Understanding of functional, integration, regression, performance, compatibility, and UAT testing.
• Experience managing defect reporting, defect triage, and QA metrics.
• Familiarity with Jira, Azure DevOps, and other test management tools.
• Knowledge of Java and C# development environments.
• Strong written, verbal, and presentation skills.
• Excellent coordination, multitasking, and stakeholder management abilities.
• Strong analytical and problem-solving skills with attention to detail.

Desirable Skills & Certifications

• Experience working within a large enterprise or consultancy environment.
• Direct exposure to testing Agentic AI solutions and autonomous AI workflows.
• Experience in performance and load testing of AI applications.
• Knowledge of data governance and data quality practices within AI ecosystems.
• ISTQB Foundation Level Certification (or equivalent experience).
• ISTQB Certified Agile Tester qualification.
• ISTQB Certified AI Tester qualification.

Preferred Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field.
• Experience supporting enterprise-scale AI transformation programs.
• Familiarity with cloud-based AI platforms and modern QA practices.

Key Competencies

• QA Delivery Management
• Risk & Defect Management
• Stakeholder Coordination
• AI Testing & Validation
• Automation & DevSecOps Collaboration
• Communication & Reporting
• Problem Solving & Analytical Thinking
• Attention to Detail