Psychology Quality Assurance Lead

About the role

This is a remote contractor position paid on an hourly basis. The Psychology Quality Assurance Lead will oversee quality control, consistency, and trainer performance across psychology-oriented AI training initiatives. The work centers on reviewing AI-produced psychology material and QA/trainer outputs, checking them against project rules, giving clear written feedback, and helping contributors stay aligned with quality expectations.

The role involves assessing accuracy in psychological content, correct use of theory, research understanding, ethical sensitivity, mental-health safety, careful diagnostic language, clarity, formatting, instruction compliance, and adherence to project rubrics. You will also identify repeated quality problems, communicate updates to trainers and QAs, support onboarding, keep documentation current, and help re-engage contributors who are inactive.

This opportunity is with a fast-scaling AI data services company that produces training data for major AI companies and foundation-model labs. The work contributes to improving AI systems by ensuring psychology-related training data is accurate, responsible, research-informed, and aligned with client needs.

There is no active project available immediately for this position. However, qualified candidates may be contacted first when relevant work becomes available and may also gain access to future projects through the expert network.

The selection process includes an AI interview, a domain-specific task, and a recruiter interview.

Responsibilities

• Review psychology items on a regular basis, identify quality concerns, share feedback directly with contributors, and escalate repeated or serious issues.
• Assess AI-generated psychology explanations, mental-health content, research summaries, behavioral guidance, theory application, and safety-sensitive responses for correctness and suitability.
• Keep trainers and QAs informed through Discord about updated guidelines, workflow changes, quality expectations, and psychology-specific review standards.
• Answer trainer and QA questions promptly and clearly, especially on psychological concepts, evidence standards, ethical limits, diagnostic caution, safety, and rubric interpretation.
• Reach out to inactive or underperforming contributors, encourage reactivation, track follow-ups, and flag availability concerns when necessary.
• Build and maintain documentation such as style guides, trackers, FAQs, quality notes, examples, honeypots, calibration tasks, and onboarding resources.
• Organize and conduct onboarding and training sessions for trainers and QAs to explain project goals, workflows, rubrics, quality standards, and psychology-specific review expectations.
• Ensure trainers and QAs apply review guidelines consistently as project requirements evolve.
• Identify unsafe, overconfident, stigmatizing, pseudoscientific, diagnostically inappropriate, or crisis-insensitive psychology content and escalate it appropriately.
• Look for recurring quality patterns, suggest process improvements, and help establish scalable QA systems for psychology AI training work.

Requirements

• A Bachelor’s, Master’s, PhD, PsyD, or equivalent professional background in Psychology, Clinical Psychology, Counseling, Behavioral Science, Cognitive Science, Neuroscience, Social Work, psychiatry-adjacent research, Education, or a closely related discipline.
• Excellent English communication skills, with the ability to understand instructions, collaborate with teams, and produce precise written feedback.
• At least 3 years of experience in psychology research, clinical or counseling environments, behavioral science, teaching, academic writing, mental-health content review, psychological assessment, or similar work.
• Strong understanding of psychological theory, research methods, cognitive and social psychology, developmental psychology, abnormal psychology, ethics, evidence-based practice, and mental-health safety.
• Ability to evaluate content against detailed rubrics and recognize overdiagnosis, unsafe advice, unsupported claims, weak study interpretation, pseudoscience, stigmatizing language, and inappropriate therapeutic recommendations.
• Preferred familiarity with CBT, developmental psychology, personality, trauma-informed communication, behavioral interventions, mental-health literacy, research design, statistics, and ethical frameworks.
• Strong preference for candidates who have led or supported remote teams of researchers, clinicians, educators, reviewers, annotators, or QAs.
• Comfort using remote collaboration tools such as Discord, Google Sheets, Google Docs, trackers, dashboards, and project management platforms.
• Highly organized and detail-focused, with experience maintaining style guides, FAQs, trackers, onboarding materials, calibration tasks, and documentation.
• Experience with AI training, data annotation, LLM evaluation, psychology QA, mental-health content review, or rubric-based reviewing is a strong advantage.

Additional information

This role is a contract position and is remote. It is described as hourly, with no immediate project currently assigned. Qualified experts may be contacted for future opportunities through the company’s expert network.

Selection process

Candidates will go through three stages: an AI interview, a domain-specific task, and a recruiter interview.