AI Content Red Team Analyst - Trust and Safety

Role overview

You will join the Trust & Safety GenAI & Emerging Products team, which focuses on helping generative AI models and applications launch safely through robust testing, risk discovery, and mitigation support. The AI Content Red Team function performs adversarial testing against generative AI products and models to surface newly emerging risks, working alongside structured evaluation efforts.

This role is centered on attacker-style testing, identifying weak points, and turning ambiguous findings into clear operational guidance that can shape product choices, policy updates, safety mitigations, and future evaluation methods. The work spans multiple product experiences, modalities, abuse scenarios, and global business contexts.

You will collaborate with Trust & Safety partners across policy, product, engineering, data science, and operations, as well as broader business teams across markets. Success in this position depends on sound judgment, creativity, analytical discipline, and the ability to convert uncertain observations into practical recommendations.

Key responsibilities

• Run structured adversarial tests on AI models, features, and policies to uncover vulnerabilities and emerging safety risks.
• Examine product behavior across different contexts and user journeys to find failure modes that may not appear in standard evaluations.
• Research jailbreaks, evasive behavior, prompt-driven attacks, and other adversarial methods relevant to content safety.
• Record findings in a clear, repeatable way, including risk summaries, reproduction steps, severity ratings, and suggested mitigations.
• Work with policy, product, and business stakeholders to confirm mitigations and close root causes.
• Contribute to the creation and upkeep of testing playbooks, taxonomies, and internal documentation resources.
• Track new adversarial tactics such as deepfakes, multimodal manipulation, and coordinated abuse, along with broader changes in the external threat landscape.

Required qualifications

• At least 3 years of experience in Trust & Safety, cybersecurity, risk or adversarial testing, or a related area.
• Hands-on experience with prompt testing, jailbreak analysis, LLM evaluation, or adversarial QA.
• Working knowledge of AI safety concerns such as jailbreaks, hallucinations, bias, and misuse patterns.
• Strong interest in generative AI safety and how AI systems can be manipulated under adversarial conditions.
• Proven ability to investigate unclear problems independently, spot subtle failure modes and abuse patterns, and draw evidence-based conclusions.
• Strong written and verbal communication skills.
• Ability to juggle multiple priorities and collaborate effectively with cross-functional teams.

Preferred experience

Experience using agentic AI tools to expand impact, including building or operating AI-enabled tools that improve speed and effectiveness of processes.

Trust & Safety work environment

The content handled in this role can include images, video, and text connected to everyday life, but may also involve bullying, hate speech, child safety issues, depictions of self-harm or harm to others, and harm to animals. As a result, day-to-day exposure to harmful material is possible.

This type of work can be rewarding, but it may also be psychologically demanding and emotionally taxing. The hiring organization shares these realities up front so candidates can make an informed decision before joining.

Employee wellbeing is treated as a priority, with support programs designed to promote physical and mental health throughout the employee journey. A collaborative and consultative approach is used to support a person-centered and integrated wellbeing experience.

About the company

Founded in 2012, the company’s mission is to inspire creativity and enrich life. Its product portfolio includes more than a dozen offerings such as TikTok, Lemon8, CapCut, and Pico, along with China-specific platforms including Toutiao, Douyin, and Xigua. These products are designed to make it easier and more enjoyable for people to connect with, consume, and create content.

Why join

The company places creativity at the center of its mission. Its products are built to help people express themselves authentically, discover new ideas, and connect with others, supported by a global and diverse workforce.

The culture emphasizes curiosity, humility, impact, and continuous improvement. Teams follow an iterative, “Always Day 1” mindset to drive meaningful breakthroughs for employees, the company, and users.

Diversity and inclusion

The organization is committed to building an inclusive workplace where people are valued for their skills, experience, and perspective. It serves a global community and aims to reflect that diversity internally by creating an environment where different voices and communities are respected.

Additional information

This is a full-time, on-site position based in Singapore, Singapore.