Senior Data Annotator

About the Role

Rex.zone invites applications for a Senior Data Annotator to enhance AI and ML model training through meticulous data labeling and review efforts. This role is crucial in delivering top-quality, guideline-compliant annotated datasets for natural language processing, computer vision, and content safety applications.

Key Responsibilities

• Lead advanced-level annotation and review processes across text, image, and multimodal data collections.
• Conduct Reinforcement Learning from Human Feedback (RLHF) preference ranking and evaluate large language models following strict rubrics.
• Execute quality assurance audits, calibration exercises, and continuous monitoring to uphold annotation standards.
• Detect and report gaps in guidelines, catalog complex edge cases, and suggest improvements to rubric clarity.
• Monitor consistency among annotators and resolve discrepancies via calibration protocols.
• Participate in prompt evaluations assessing helpfulness, safety, and adherence to policies.
• Engage in content safety labeling and the management of sensitive topic classifications.
• Collaborate with engineering and data operations teams to enhance annotation tools, optimize workflows, and improve error identification.

Required Qualifications

• Proven experience in data annotation, labeling, or quality assurance for machine learning datasets.
• Aptitude to rigorously apply detailed rubrics and maintain compliance with annotation guidelines.
• Understanding of RLHF processes, preference data appraisal, and large language model evaluation techniques.
• Strong written communication skills necessary for documenting judgments and complex cases.
• Hands-on experience with NLP tasks such as named entity recognition and/or computer vision annotation methods like bounding boxes and segmentation.

Quality Metrics

Performance is evaluated through regular audits, inter-annotator agreement assessments, rubric adherence, calibration rounds, and error rate analysis to ensure the highest integrity of training data for downstream models.

Remuneration

The position offers a competitive hourly wage ranging from $30 to $50 USD.