LLM AI Quality Analyst (Personalization)

Role overview

This short-term remote contract is for an English-speaking AI quality analyst focused on personalization. The assignment runs for one month, requires 20 hours per week, offers $11.5 per hour, and is available for an immediate start.

Key responsibilities

• Create multi-turn conversation prompts that use personal context appropriately.
• Review personalized AI answers for relevance, grounding, and usefulness.
• Check whether the model uses personal information correctly or incorrectly.
• Compare two responses side by side and rank them based on quality.
• Spot grounding issues, weak assumptions, and overdone personalization.
• Draft concise, well-structured explanations that point to exact conversation turns.
• Capture and validate model debugging details and evidence of data-source usage.
• Keep evaluation data clean by deleting conversation records after review.

Required profile

• Strong command of English for both reading and writing.
• Background in data annotation, AI evaluation, content moderation, or a similar field.
• Good analytical judgment and a careful eye for detail.
• Comfort with reviewing subtle, uncertain, or ambiguous AI outputs.
• Willingness to use a primary personal Google account with data sources enabled.
• BS/BA degree or comparable experience in an analytical discipline.
• Ability to communicate feedback clearly in a structured format.
• Capacity to work independently in a remote environment.
• Reliable computer and stable internet connection.

Application steps

The selection process includes submitting the application form, completing the ICF, and taking an assessment.