Evals Engineer

Overview

Zof AI is looking for a dedicated Evals Engineer to join their Accra, Ghana office full-time. This role focuses on developing comprehensive tests to verify the effectiveness of AI products beyond superficial demos, ensuring that AI systems perform as intended with rigorous evaluation criteria. The position combines expertise in quality assurance with deep domain knowledge to build evaluation suites, verification tools, and quality controls that ensure product correctness.

Key Responsibilities

• Create evaluation suites specifically tailored for AI and agent-based systems.
• Develop verification frameworks that validate AI outcomes against expected results.
• Establish quality gates that determine the readiness of features for release.
• Translate domain knowledge and user needs into measurable and testable validation checks.
• Identify and diagnose failure patterns such as regressions, hallucinations, and hidden errors.
• Make evaluation feedback comprehensible and actionable to engineers, product teams, and clients.
• Integrate evaluation processes within continuous integration and development workflows.
• Elevate the company-wide standards for defining what constitutes a successful AI system.

Candidate Requirements

• Proven experience in testing, evaluating, or quality assurance of sophisticated software systems.
• In-depth understanding of common failure modes in large language models (LLMs) and agent systems.
• Strong analytical mindset paired with a healthy skepticism toward unverified performance claims.
• Proficiency in coding to develop testing harnesses and automation scripts.
• Meticulous attention to detail with a commitment to maintaining high-quality standards.
• Effective written and verbal communication skills.
• Ability to thrive in a fast-paced, rapidly evolving environment.
• Demonstrated high level of ownership and initiative.

Preferred Qualifications

• Hands-on experience creating evaluation benchmarks or test infrastructure for LLMs.
• Background in QA engineering, SDET roles, or automated testing methodologies.
• Subject matter expertise in areas where correctness is critical.
• Familiarity with statistical methods for performance evaluation.

Compensation and Benefits

The salary for this role ranges from GHS 6,500 to GHS 13,000 per month and includes meaningful equity participation. Employees receive a comprehensive living package that may include housing support or provided accommodation, meals prepared by a private chef, equipment like a MacBook Pro, and wellness amenities. Additional benefits available depending on eligibility include laundry and cleaning services, access to advanced AI development tools such as Cursor Ultra, Claude Code Ultra, and OpenAI Codex Max, recreational activities including basketball, table tennis, and foosball, as well as massage and fitness support.

Location and Work Environment

This full-time role is based onsite in Accra, Ghana. The environment encourages collaboration in a dedicated workspace that may include opportunities for communal living arrangements designed to foster focus and innovation.