AI Tester (m/f/d)

AI Tester (AI Validation & RAG Testing)

We are looking for an AI Tester to join a large enterprise environment, focusing on the validation and quality assurance of AI-powered applications. Your primary responsibility will be to assess AI model behavior, confirm the accuracy of responses, and identify potential risks like hallucinations and model drift. This position is vital for ensuring that AI systems are dependable, scalable, and ready for production deployment across various applications.

Key Responsibilities

• Collaborate with application development teams to test and validate AI-driven solutions.
• Conduct RAG evaluations, specifically identifying hallucinations, inaccuracies, and grounding problems.
• Monitor and detect model drift and ensure response consistency over time.
• Perform functional and exploratory testing on AI-generated outputs.
• Assist with stress and load testing by analyzing system responses under heavy query loads.
• Validate AI responses against established test datasets and performance benchmarks.
• Work with engineering teams to enhance model performance and the quality of outputs.
• Contribute to the development of AI testing frameworks, metrics, and validation strategies.

Required Skills & Qualifications

• Experience in AI testing, quality assurance, or data validation.
• A solid grasp of Large Language Models (LLMs), RAG systems, and AI behavior.
• Proven ability to identify hallucinations, bias, and inconsistencies in AI models.
• Familiarity with API testing tools such as Postman.
• Understanding of performance and stress testing principles.
• A strong analytical and problem-solving aptitude.
• Capability to thrive in cross-functional Agile team settings.

Additional Information

AI Tester position located in Abu Dhabi, United Arab Emirates.