Language Model Analyst - Remote Part-time Role

About The Role

Mercor, a San Francisco-based company backed by prominent investors like Benchmark, General Catalyst, and Jack Dorsey, connects top-tier creative and technical professionals with leading AI research labs. They are seeking a Language Model Analyst specializing in English and Tamil for a contract, remote, part-time assignment.

Key Responsibilities

• Verify factual accuracy by consulting reliable public resources and external verification tools.
• Produce detailed, high-quality human evaluation data by identifying strengths, deficiencies, and factual errors in model responses.
• Evaluate the reasoning, clarity, tone, and thoroughness of responses generated by language models.
• Ensure generated replies conform to desired conversational standards and internal guidelines.

Qualifications

• Bachelor's degree is mandatory.
• Native fluency in Tamil is required.
• Substantial prior experience working with large language models.
• Exceptional written proficiency in English.
• Strong attention to detail and analytical skills.
• Background in research, policy, analytics, linguistics, or engineering—or other fields involving structured analytical processes.

Preferred Experience

• Familiarity with Reinforcement Learning from Human Feedback (RLHF), model evaluation, or data annotation projects.
• Experience in producing or editing high-caliber written content.
• Skill in comparing multiple outputs and providing nuanced qualitative assessments.

Application Process

• Submit a resume upload.
• Complete an AI-driven interview based on your submitted resume.
• Fill out a submission form.

Additional Notes

• Applications are reviewed daily; completion of all steps is required for consideration.
• For further details about the interview process and platform, candidates may contact the support team via support@mercor.com.