Model Data Operations Specialist - Evaluation & Judgement Center, Platform Responsibility

About the Team

The Platform Responsibility team is pioneering innovation in content safety by deeply integrating AI technologies. They focus on building next-generation infrastructures using advanced Large Language Models (LLMs) to transform risk identification and governance processes. Through developing high-quality data pipelines and embedding AI in key decision-making processes, the team drives substantial gains in both efficiency and accuracy. Positioned at the crossroads of scalable technology and engineering, they support global business expansion while upholding user trust.

Role Overview

This role supports TikTok's Community Guidelines by utilizing LLM capabilities and prompt engineering to develop a sophisticated large model Agent that embodies "absolute correctness," enabling stable and reliable evaluation of content safety. The role involves leading efforts to refine operational strategies and optimize prompts to enhance the Agent's accuracy in detecting risky content.

Key Responsibilities

• Carry out AI-focused special projects linked to product R&D such as annotating datasets for LLM training, evaluating model iteration performance, researching industry applications, and ensuring security compliance.
• Collaborate intensively with upstream product and technology teams to meet business goals effectively.
• Standardize content risk definitions by establishing access criteria and translating qualitative policy documents into concrete annotation SOPs applicable across diverse risk domains including pornography, violence, terrorism, politics, and abuse.
• Manage the delivery of Gold Standard datasets encompassing demand analysis, acceptance, solution customization, training, quality control, and deployment.
• Develop and operate an adjudication mechanism to maintain consistent judgments on assorted risk levels within internal cases.
• Design and continually enhance rule routing logic aligned with LLM content safety workflows and community guidelines, aiming to improve model effectiveness and optimize resource utilization.
• Build and oversee an AI content quality management framework responsible for quality metrics, while continuously engaging in problem diagnosis, data analysis, and model refinement.

Minimum Qualifications

• Bachelor’s degree or higher with at least three years of experience in model operations, AI/ML operations, or related fields.
• Proven expertise in leading community policy formulation and collaborating cross-functionally with Product, Engineering, and Operations teams.
• Strong analytical and problem-solving skills, capable of identifying root issues, prioritizing projects, and implementing structured improvement plans with measurable outcomes.
• Keen interest in internet products and familiarity with contemporary AI trends and mainstream LLMs (e.g., ChatGPT, Gemini), plus hands-on experience in model training, evaluation, or prompt engineering.
• Data-driven mindset with robust analytical skills to convert insights into practical strategies and resolve operational or model-related challenges proactively.
• Sound understanding of community safety and risk governance frameworks, with demonstrated risk-awareness and ability to design scalable mitigation solutions.

Preferred Qualifications

• Experience or background in product management or technical roles within search systems, AI, or large language models is advantageous.
• Self-driven and result-oriented, with strong influencing skills suited for dynamic, ambiguous environments.

About TikTok

TikTok is the premier platform for short-form mobile videos, dedicated to inspiring creativity and delivering joy worldwide. With headquarters in Los Angeles and Singapore and offices around the world, TikTok connects diverse global communities through engaging and innovative content.

Why Join Us

TikTok values creativity and fosters an inclusive environment where individuals can authentically express themselves and connect globally. The team thrives on curiosity, humility, and impact orientation, embracing challenges as opportunities to advance technology and community value. We uphold an "Always Day 1" culture to continuously innovate and grow together.

Diversity & Inclusion

Committed to diversity, TikTok cultivates a workplace that respects varied perspectives and experiences. We believe this inclusivity drives our mission to inspire creativity and bring joy to a broad spectrum of communities worldwide.