Language Model Evaluator (English & Punjabi) - Remote Part-time
Remote · Part Time
Be the first to apply
- Experience
- Any
- Salary
- USD 15 – USD 20 / hour
- Openings
- 1
- Posted
- 2 weeks ago
- Work mode
- Work from home
- Education
- Bachelor's degree
- Resume
- Required to apply
Job description
About the Role
Mercor is a company that connects exceptional creative and technical professionals with top-tier AI research institutions. Based in San Francisco, Mercor is backed by prominent investors including Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.
This position is a contract role for a Generalist focusing on English and Punjabi languages, with fully remote work arrangements and compensation ranging from $15 to $20 per hour.
Key Responsibilities
- Perform fact verification utilizing reliable public references and external verification tools.
- Produce detailed human evaluation feedback by highlighting strengths, pinpointing areas needing improvement, and noting any factual errors in AI-generated responses.
- Evaluate the quality of reasoning, clarity, tone, and overall completeness of responses.
- Confirm that AI model outputs conform to expected conversational norms and system guidelines.
- Manage tasks independently and asynchronously while adhering to deadlines to enhance AI model capabilities.
Qualifications and Experience
- Possession of a Bachelor's degree.
- Native proficiency in Punjabi language.
- Extensive experience with large language model (LLM) applications.
- Strong command of written English.
- Exceptional attention to detail.
- Background or experience in fields requiring structured analytical skills.
Preferred Skills
- Familiarity with Reinforcement Learning from Human Feedback (RLHF), model evaluation processes, or data annotation practices.
- Experience in producing or editing high-quality written content.
- Capability to compare multiple AI outputs critically and make nuanced qualitative assessments.
Application Process
- Submit your resume.
- Complete an AI-based interview derived from your resume in approximately 20-30 minutes.
- Fill and submit the application form.
Additional Information
- Detailed information on the interview process and platform use is available through Mercor's resource documentation.
- For support or inquiries, applicants can contact support@mercor.com.
- The recruitment team reviews applications daily; completing all application steps is necessary for consideration.