Job description

Role overview

This is a short-term remote contract for an LLM AI Quality Analyst focused on personalization. The position requires a commitment of 20 hours per week, lasts for 1 month, pays $11.5 per hour, and is set to begin immediately.

Core responsibilities

Create multi-step conversational prompts that use personal context appropriately.
Review personalized AI outputs for usefulness, relevance, and whether they stay grounded in the provided information.
Check when the model uses personal data correctly or incorrectly in its answers.
Carry out side-by-side comparisons of responses and rank the better output.
Spot grounding issues, weak inferences, and overly forced personalization.
Prepare concise, well-organized explanations that point to the specific conversation turns involved.
Pull and confirm model debug details and the sources of data used by the system.
Follow strict data-handling practices, including removing evaluation conversations after use.

Requirements

Strong command of English for both reading and writing.
Prior experience in data annotation, AI evaluation, content moderation, or a similar analytical role.
Good judgment, careful analysis, and strong attention to detail.
Comfort evaluating subtle, unclear, or ambiguous AI responses.
Ability to work with a primary personal Google account with data sources enabled.
A BS/BA degree, or equivalent practical experience in a relevant analytical area.
Clear written communication skills and the ability to give feedback in a structured way.
Self-driven approach with the ability to work independently from a remote setup.
A dependable desktop or laptop with a stable internet connection.

Application process

Applicants will need to complete the application form, the ICF, and an assessment.

Terms

This is a remote contract role based in Ireland. The workload is 20 hours per week, the engagement lasts 1 month, and the start date is immediate.

LLM AI Quality Analyst (Personalization) English