AI Tester (AI Validation & RAG Testing)

Role overview

This position is for an AI Tester who will help verify the quality and reliability of AI-enabled applications in a large enterprise setting. The focus is on assessing model behaviour, checking response correctness, and spotting issues such as hallucinations and model drift before these systems are used in production.

The role plays an important part in making sure AI solutions are dependable, scalable, and suitable for real-world business use across different scenarios.

Core responsibilities

• Partner with application teams to assess and validate AI-based solutions.
• Carry out RAG testing, with attention to hallucinations, incorrect outputs, and weak grounding.
• Track model drift and observe whether responses remain stable over time.
• Run functional and exploratory checks on AI-generated responses.
• Assist with stress and load testing by reviewing behaviour under heavy query traffic.
• Compare model outputs with test data and expected benchmark results.
• Work with engineering teams to improve output quality and model performance.
• Help shape testing approaches, evaluation metrics, and validation standards for AI systems.

Required background and skills

• Prior experience in AI testing, quality assurance, or data validation work.
• Good understanding of large language models, RAG architectures, and AI response patterns.
• Experience spotting hallucinations, bias, and inconsistent model behaviour.
• Exposure to API testing utilities such as Postman.
• Awareness of performance and stress testing principles.
• Strong analytical thinking and troubleshooting ability.
• Comfort working in cross-functional Agile teams.

Additional information

Location: Abu Dhabi Emirate, United Arab Emirates.

Employment type: Full-time, onsite.