Domain Expert - Geography

Role Overview

We are looking for an expert in Geography to enhance and assess Large Language Models (LLMs). The candidate will craft sophisticated prompts and critically evaluate AI responses within areas such as physical and human geography, GIS, climate science, geopolitics, and demographics. Identifying discrepancies, reasoning flaws, and areas lacking knowledge will be key components of the job.

Key Responsibilities

• Design comprehensive prompts encompassing geography-related domains including GIS, climate, and geopolitics.
• Assess AI-generated outputs for factual correctness, logical coherence, thoroughness, and subtlety.
• Detect hallucinations, logical errors, outdated data, and challenging edge scenarios.
• Create benchmark datasets and adversarial tests to stress model reliability.
• Offer feedback supported by credible references and evidence.
• Work closely with AI researchers to boost model accuracy and capabilities.
• Ensure meticulous quality in annotations and maintain thorough documentation.

Day-to-Day Activities

• Compose domain-centric prompts at different complexity levels.
• Compare multiple AI answers and rank them based on quality.
• Justify the correctness or flaws in responses using authoritative sources.
• Identify ambiguous queries and propose enhanced prompt formulations.

Qualifications

• Master's degree or higher in Geography or related fields.
• At least three years of professional, research, or academic experience in the discipline.
• Strong command of written English paired with solid analytical capabilities.
• Keen attention to detail is essential.

Preferred Skills

• Familiarity with LLMs, generative AI, prompt engineering, or AI evaluation processes.
• Academic publications, recognition in the industry, or teaching background favorably regarded.
• Capability to work autonomously delivering impartial, research-backed evaluations.