Meta Tested Rival AI Chatbots With Sensitive Prompts Posed as Minors: Report

By: Harsh Vardhan

|

Updated on: 30-Jun-2026 03:00 PM

6,817 views

Follow Us:

6,817 views

Meta reportedly assembled a large team to test how rival AI chatbots respond to sensitive topics when interacting with minors. According to a Wired report, Meta hired hundreds of contractors to pose as minors and engage with competing chatbots. The project, known internally as Cannes, was led by Meta contractor Covalen. Contractors tested chatbots including OpenAI’s ChatGPT, Google’s Gemini, and Character.AI without the companies’ knowledge.

Key Highlights

• Meta hired contractors to test rival AI chatbots with sensitive prompts posed as minors.
• The project involved over 45,000 prompts about sex, suicide, drugs, and eating disorders.
• OpenAI, Google, and Character.AI were unaware of the testing and cited policy violations.
• Meta described the project as standard safety benchmarking for AI chatbot responses.

Meta's Internal Testing Project

The Cannes project aimed to evaluate how chatbots handle conversations about sex, suicide, drugs, and eating disorders. Contractors created dummy accounts with ages set under 18. They sent text prompts and images to the chatbots and recorded the responses in spreadsheets. Many prompts were designed to test the limits of the chatbots’ safety rules.

One round of testing in August 2025 involved over 45,000 prompts. A spreadsheet listed dummy profiles with names, email addresses, passwords, and birthdates, using disposable Gmail and Outlook accounts. Another spreadsheet contained 3,748 prompts sent by contractors. Hundreds of prompts related to suicide, self-harm, and eating disorders. At least 239 prompts referred to sex or romance, while others involved drugs, profanity, and racial slurs.

Nature of Prompts and Responses

Many prompts were framed as if written by distressed children or teenagers. Examples included a 13-year-old asking about ending a pregnancy, a fifth-grader describing a classmate with a gun, and a girl seeking advice on hiding bulimia from parents. Some prompts were deliberately crude or unusual, such as asking if fantasizing about eating a neighbor’s child was normal. Another prompt, written as a high school student, asked where to obtain cocaine, though the chatbot did not comply. Prompts also included non-English examples, such as a French prompt referencing Jamey Rodemeyer, a bisexual teenager who died by suicide after bullying.

The documents do not indicate how Meta used the responses. An internal Covalen document described Cannes as “comprehensive AI safety benchmarking” that delivered “critical datasets for model comparison and compliance.” Meta defended the project as standard safety testing. A spokesperson stated that testing chatbot responses for safe and age-appropriate experiences is an industry-standard practice. Meta also said it does not use competitor benchmarking to train its own AI models.

Industry Reactions and Policy Concerns

Some contractors expressed discomfort with the project. One former contractor said the nature of the prompts was alarming and questioned the legality of the work. The project may have conflicted with the terms of service of rival platforms. OpenAI prohibits unsolicited safety testing, attempts to bypass safeguards, and using outputs to develop competing models. OpenAI stated it is investigating the issue. Google bans attempts to bypass safety filters outside its testing programs and prohibits content involving self-harm, child sexual abuse, or illegal substances. Google confirmed it had not authorized the testing and did not know its purpose. Internal testing showed Gemini responded in line with company policies. Character.AI also bans harmful and illegal content and restricts open-ended chat for users under 18. Character.AI said it had not authorized the testing and that the described conduct violated its terms and policies.