Psychology Quality Assurance Lead

Role overview

This is a remote contractor position for a Psychology Quality Assurance Lead who will safeguard the quality, consistency, and readiness of psychology-focused AI training work. You will review both AI-generated psychology content and the work of trainers and QA contributors, making sure outputs align with project rubrics, ethical expectations, and safety standards.

The position supports a fast-scaling AI data services business that creates training data for major AI companies and foundation-model labs. Your contribution will help improve AI systems by keeping psychology-related content accurate, careful, research-informed, and suitable for client use.

This opportunity is hourly in nature and remote. There is not an active project assignment available right away, but suitable candidates may be contacted first when matching projects open up and may also gain access to future work through the expert network.

What you will do

• Monitor work quality by sampling psychology tasks, identifying issues, giving direct feedback, and escalating recurring or serious concerns.
• Review psychology content produced by AI, including explanations, mental-health material, research summaries, behavior guidance, theory use, and safety-sensitive replies.
• Check work against detailed rubrics, with attention to accuracy, clarity, formatting, instruction-following, and project-specific standards.
• Evaluate outputs for psychological correctness, proper use of theory, research literacy, ethical judgment, diagnostic caution, and mental-health safety.
• Communicate updates to trainers and QA contributors through Discord, including guideline changes, workflow updates, and expectations for quality review.
• Answer questions from trainers and QAs about psychology concepts, evidence quality, ethical limits, safety concerns, and how to interpret rubrics.
• Reach out to inactive or inconsistent contributors, encourage re-engagement, and track availability or follow-up issues.
• Build and maintain project documentation such as style guides, trackers, FAQs, examples, calibration tasks, honeypots, and onboarding materials.
• Plan and run onboarding or training calls to walk trainers and QAs through project requirements, review standards, and workflow expectations.
• Keep quality practices aligned across the team so reviewers apply the same standards as projects evolve.
• Identify unsafe, stigmatizing, pseudoscientific, overconfident, diagnostically inappropriate, or crisis-insensitive content and flag it promptly.
• Suggest process improvements and help create scalable QA workflows for psychology-related AI training projects.

Requirements

• A bachelor’s, master’s, PhD, PsyD, or equivalent professional background in psychology, clinical psychology, counseling, behavioral science, cognitive science, neuroscience, social work, psychiatry-adjacent research, education, or a closely related discipline.
• Strong command of English for reading project instructions, coordinating with teams, and writing clear feedback.
• At least 3 years of relevant experience in psychology research, clinical or counseling settings, behavioral science, teaching, academic writing, mental-health content review, psychological assessment, or similar work.
• Solid understanding of psychology theory, research methods, cognitive and social psychology, developmental psychology, abnormal psychology, ethics, evidence-based practice, and mental-health safety.
• Ability to judge content against detailed review criteria and detect overdiagnosis, unsafe advice, unsupported claims, weak study interpretation, pseudoscience, stigmatizing wording, and unsuitable therapeutic guidance.
• Working knowledge of CBT, developmental psychology, personality, trauma-informed communication, behavioral interventions, mental-health literacy, research design, statistics, and ethical guidelines is preferred.
• Prior experience guiding or supporting remote teams of researchers, clinicians, educators, reviewers, annotators, or QAs is highly desirable.
• Comfort using Discord, Google Sheets, Google Docs, trackers, dashboards, and project management tools in a fast-paced remote setting.
• Strong organization and attention to detail, with the ability to manage style guides, FAQs, trackers, onboarding resources, and calibration materials.
• Experience with AI training, data annotation, LLM evaluation, psychology QA, mental-health content review, or rubric-based review will be an added advantage.

Additional information

Selection will include an AI interview, a domain-focused task, and a recruiter interview.

This role is contract-based, remote, and hourly.

The company has a Melbourne, Victoria, Australia location listed, but the work itself is remote.

No immediate project is available at the moment; the role is intended to keep qualified experts ready for upcoming opportunities.