English Speaking Generalist

Role Overview

This position involves evaluating large language models through the creation and comparison of various prompts while working remotely. Candidates will interact with AI platforms, providing in-depth verbal feedback in American English and ensuring adherence to evaluation standards.

Key Responsibilities

• Develop diverse prompts on a range of subjects to effectively challenge AI language models.
• Conduct parallel testing by submitting identical prompts to ChatGPT and Claude, while recording both screen activity and audio commentary.
• Analyze and articulate differences between model outputs, providing real-time verbal evaluations.
• Deliver clear verbal reasoning and preferences using fluent American English.
• Interpret project documentation to maintain consistent evaluation criteria and ensure precise task execution.
• Collaborate within project frameworks to aid the ongoing improvement of AI models.

Candidate Requirements

• Exceptional command of spoken and written American English with clear enunciation.
• Experience interacting with large language models like ChatGPT, Claude, or equivalents.
• Knowledge or familiarity with AI evaluation methodologies.
• Comfortable performing extended screen and audio recording sessions.
• Prior exposure to data annotation, AI model training, quality assurance, content analysis, or research preferred.
• Possess a robust technical setup including a dependable computer and stable internet connection.

Application Details

• Application is straightforward and begins with submission.
• Successful candidates will receive follow-up emails outlining subsequent steps.
• An interview and resume assessment will be part of the selection process.