Meta Contractors Posed as Teens to Test Rival Chatbots
Meta contractors posed as teens to prompt rival chatbots on high-risk subjects.
"Meta's contractors went undercover as teens to push rival chatbots to their limits. Talk about extreme testing!"
Hundreds of contractors working for Meta, managed by Covalen, posed as minors to test rival chatbots like OpenAI's ChatGPT, Google's Gemini, and Character.AI. This project, known internally as Cannes, involved creating dummy under-18 accounts and sending prompts and images related to suicide, sex, and drugs. The goal was to assess how these chatbots would respond to high-risk subjects, with some prompts designed to challenge their safety systems.
The testing involved sending over 45,000 prompts in a single round, with many focusing on suicide, self-harm, eating disorders, sex, and profanity. Contractors used images of pills, knives, and nooses. Some prompts were written from the perspective of children or teenagers in crisis, such as a 13-year-old asking about ending a pregnancy or a high school student inquiring about cocaine. The companies behind the targeted chatbots were unaware of this testing.
Meta defended the project as routine safety testing and industry-standard practice for ensuring safe and age-appropriate experiences. An internal Covalen document described it as "comprehensive AI safety benchmarking" providing "critical datasets for model comparison and compliance." Meta stated it does not use competitor benchmarking to train its own AI models.
This highlights the aggressive and potentially ethically questionable tactics companies employ to benchmark AI safety. Businesses developing or utilizing AI should be aware of such testing methods and their implications for data privacy and ethical AI development. It underscores the ongoing challenges in ensuring AI safety and preventing misuse.
Relevant tools
Find the right AI tool for your business
Chat with Insta and get matched to the right tool in seconds.
Try Insta Tool Finder →