AI Data Annotator / AI Trainer

About ProtocolClear

ProtocolClear is an independent consultancy led by principals, focused on security, privacy, AI governance, and compliance for clients in regulated industries across the United States and Nigeria. The team is expanding an internal data and model evaluation function to label and assess the information used to train and test AI systems. This is a remote role for a Lagos-based annotator who wants stable, serious work and the opportunity to develop with a fast-moving team. The work demands careful, standards-based handling for clients where data quality and discretion matter greatly.

Role overview

You will annotate and evaluate data across multiple formats and problem types, including text classification, entity and span tagging, image bounding boxes, segmentation, and reinforcement-learning-from-human-feedback tasks. In the RLHF work, you will compare model outputs, score them using a rubric, and explain the reasoning behind your judgment. The subject matter varies, so the role is about annotation quality and consistency rather than expertise in one domain. Performance is measured by accepted batches, accuracy, agreement, and gold-task results, not by hours spent online. You will operate within a defined workflow that includes instructions, gold sets, and calibration checks, and you are expected to deliver at a professional level from the start.

Quality expectations

Every submission is reviewed against a written definition of done before it is approved and paid. Hidden gold items are used to assess accuracy, with a target of at least 95% on those checks. Agreement with consensus and expert review is measured through kappa, with an expectation of at least 0.80 on objective tasks. Labels must stay tight and consistent, without drift. For RLHF tasks, rationales must be clear, specific to the item, and structured around the rubric one criterion at a time, with safety or harmlessness prioritized where required. Client data must remain confidential under NDA and be handled only in approved tools. Work generated by ChatGPT, Claude, or any other AI tool is not accepted unless a task explicitly authorizes it. Any work that misses these standards is rejected and must be redone before payment is made.

What the role involves

• Use detailed guidelines and taxonomies to label text, image, and document data, including classification, entity and span tagging, and bounding-box or segmentation tasks.
• Compare and rank model responses, evaluate them against rubrics, and write well-supported explanations covering usefulness, factual accuracy, and safety or harm.
• Meet daily throughput targets while maintaining the required quality levels for agreement, gold checks, and minimal rework.
• Complete gold and honeypot checks and participate in calibration sessions to keep alignment high.
• Escalate anything unclear or outside the instructions instead of guessing, and keep a log of edge-case decisions so similar items are handled consistently.
• Work only in approved tools under NDA and apply careful handling to any personal or sensitive information.
• Stay organized with focus, self-review, and file version control so accepted work is not lost, deadlines are met, and revisions are incorporated properly.

Required profile

The ideal candidate already has hands-on experience in data annotation or AI training and understands the work beyond casual tool use. You should be able to explain concepts such as inter-annotator agreement, kappa, gold tasks, honeypots, consensus, and adjudication in your own words. Experience with RLHF or model-response evaluation is needed, including the ability to write rubric-based rationales. Strong written English, careful attention to detail, and the ability to stay focused during long labeling sessions are important. You should be comfortable working to strict written standards and with a pay model that rewards accepted, quality-passed work. Screen-proctoring may be used for designated tasks, and unauthorized AI tools must not be used. A personal laptop, dependable backup power such as an inverter or generator, and stable internet access are required; equipment is not provided, though an internet and power stipend is included.

Preferred qualities

• Ability to label quickly without sacrificing accuracy.
• Comfort following long, detailed instructions exactly.
• Experience judging or comparing AI and chatbot responses.
• Good judgment in flagging unclear or conflicting directions instead of guessing.
• Ability to sustain focus on repetitive screen-based work.
• Experience giving written feedback or brief decision justifications.
• Habit of checking work before submission to catch mistakes.
• Basic spreadsheet knowledge, including Excel or Google Sheets.
• Calm communication under deadlines and clear escalation of blockers.

Compensation and trial period

Compensation ranges from ₦500,000 to ₦1,200,000 per month and is structured as a base amount plus per-accepted-batch output pay, with performance bonuses linked to agreement, gold-task accuracy, and low rework. A monthly internet and power stipend is included. The first three months are a paid trial period paid at per-batch rates plus the stipend. After successful confirmation, the full compensation structure applies. Earnings can increase over time with strong quality and higher volume. Equipment is not supplied.

Working style

The role operates on a full-time schedule aligned to Lagos working hours. Coordination happens day to day through the project tool and WhatsApp. Each batch is checked against a written quality standard before approval. The team values early communication about blockers, on-time delivery, escalation of real ambiguity instead of guesswork, and careful, consistent output. All work produced belongs to the company and is governed by NDA.