AI QA Engineer

Role Description

We are seeking an AI QA Engineer to assure the quality, reliability and production readiness of conversational and agentic AI solutions delivered for enterprise clients. You will work across chatbot, voicebot, RAG and integrated AI applications, validating that solutions meet functional, technical and user experience requirements before release.

This role suits someone who combines strong QA discipline with a practical understanding of AI behaviours, conversation design, integrations and real-world edge cases. You will also help introduce and develop automated testing approaches to improve regression coverage, consistency and delivery speed.

Responsibilities

• Design and execute test plans, test cases and validation frameworks for conversational and agentic AI solutions

• Develop and maintain automated tests for key user journeys, regression scenarios and integration points where appropriate

• Validate chatbot and voicebot behaviours across functional flows, edge cases, fallback handling and escalation paths

• Test AI outputs for quality, consistency, accuracy and alignment to business requirements

• Support testing of RAG, prompt-driven and LLM-enabled experiences, including response relevance and failure handling

• Verify integrations with APIs, business systems, contact centre platforms and third-party services

• Identify, log, track and retest defects through to resolution

• Collaborate with AI Engineers, Architects and Project Managers across build, QA, UAT and deployment

• Contribute to release readiness by assessing stability, security, compliance and production risk

• Help improve QA standards, test assets and repeatable validation practices across projects

Ideal Profile

• Previous experience delivering AI, conversational AI or automation projects into production (preferred but not mandatory)

• Confident testing complex journeys across AI logic, workflows, integrations and user experience

• Comfortable working in a fast-moving environment where requirements evolve and challenges are part of the role

• Practical, proactive and detail-oriented, with a strong sense of ownership and quality

• Keen to solve problems, raise standards and help shape how AI solutions are tested effectively