Language Model Evaluator (English & Bengali) - Remote Part-Time Contract
Remote · Part Time
Be the first to apply
- Experience
- Any
- Salary
- USD 15 – USD 20 / hour
- Openings
- 1
- Posted
- 3 hours ago
- Work mode
- Work from home
- Education
- Bachelor's degree
- Eligibility
- Applicants must be native Bengali speakers holding a bachelor's degree and experienced with large language models. Candidates with backgrounds in analytical fields and prior exposure to RLHF or data annotation are preferred.
- Resume
- Required to apply
Job description
About the Role
Mercor offers a unique opportunity to collaborate with top-tier AI research teams through a part-time remote contract role as a Language Model Evaluator. This position primarily requires expertise in both English and Bengali languages and involves evaluating AI-generated responses for quality and accuracy to enhance language model performance.
Key Duties
- Verify facts using credible public sources and various external tools.
- Create high-quality human evaluation data by identifying strengths, weaknesses, and factual errors in AI outputs.
- Evaluate the reasoning, clarity, tone, and completeness of model responses.
- Ensure responses adhere to conversational standards and established system guidelines.
- Work independently while managing deadlines to contribute to continuous AI improvement.
Eligibility and Requirements
- Possession of a bachelor's degree.
- Native proficiency in Bengali.
- Extensive experience with large language models (LLMs).
- Strong written English communication skills.
- Exceptional attention to detail.
- Background in analytical or structured disciplines such as research, policy, analytics, linguistics, or engineering.
Preferred qualifications include prior involvement in Reinforcement Learning from Human Feedback (RLHF), model evaluation, or data annotation projects, proven experience in producing or refining high-quality written material, and expertise in comparing multiple outputs to make nuanced qualitative assessments.
Compensation and Contract Details
This contract position offers an hourly payment ranging from $15 to $20. The work arrangement is fully remote with flexible part-time hours.
Application Procedure
- Submit your updated resume.
- Complete an AI-based interview tailored to your resume.
- Fill out the necessary application form.
Applications are reviewed daily; to be considered, candidates should finalize all application stages promptly.
Additional Information and Support
- Detailed interview and platform instructions are made available by the company.
- For assistance or inquiries, support is available via email.