Domain Expert - History

Role Overview

We are looking for a proficient History Domain Reviewer to assist in the quality assurance process of Large Language Model evaluation projects. This role demands scrutinizing domain-specific prompts, assessing the completed tasks for their precision and overall quality, ensuring compliance with project protocols, and delivering actionable feedback to uphold high annotation standards. The candidate's expertise in history will be crucial in verifying the accuracy, uniformity, and excellence of AI evaluation data representing various historical subjects.

Key Responsibilities

• Examine and validate prompts related to ancient, medieval, modern, and contemporary history alongside civilizations, historical events, political and military history, archaeology, historiography, and allied areas.
• Assess completed assignments to confirm factual correctness, quality of reasoning, thoroughness, and conformity with project standards.
• Detect factual errors, inconsistencies in logic, hallucinations, outdated content, and poor-quality annotations.
• Guarantee that prompts are sufficiently challenging, pertinent, and in line with project goals.
• Offer clear, constructive, and evidence-backed feedback to contributors aimed at enhancing task quality.
• Maintain uniformity across evaluations by adhering to established quality benchmarks and review protocols.
• Report ambiguous or complex issues when necessary and accurately record review outcomes.
• Work in partnership with project managers and AI teams to continually elevate evaluation standards and review methodologies.

Minimum Qualifications

• A Master's degree or higher in History, Archaeology, Historical Studies, Political History, World History, or a closely related field.
• At least three years of professional, research, teaching, or industry experience in the history domain.
• Preferably prior experience in reviewing, evaluating, editing, or quality-controlling domain-specific content.
• Excellent written English skills combined with strong analytical capabilities.
• High attention to detail with a keen ability to spot factual, logical, and contextual inaccuracies.