Character.AI to ban children under 18 from talking to its chatbots

(Bloomberg/Rachel Metz) — Artificial intelligence startup Character.AI will ban kids from having conversations with chatbots on its platform following growing pressure from lawmakers and a raft of lawsuits alleging the company’s so-called characters are responsible for harms to children.

The company said Wednesday that by Nov. 25 it will stop users under the age of 18 from having open-ended conversations with its chatbots, which users can create and talk to. The characters range from digital representations of popular video-game heroes to mimics of celebrities like billionaire Elon Musk. Until the ban takes effect, the company said it will limit the amount of time that kids are able to talk with its chatbots; this will start at two hours per day and go down over the following weeks.

Nvidia just became the world’s first $5 trillion company.

Meta, Applied Materials reveal plans for Bay Area layoffs

Nvidia CEO prepares to unveil AI deals with Samsung, Hyundai

Amazon cuts 14,000 corporate jobs as spending on artificial intelligence accelerates

Nvidia to invest $1 billion in Nokia in AI networking push

Services like Character.AI, which allows users as young as 13 in the US, are popular with teens. According to a survey by nonprofit Common Sense Media, more than 70% of teens have used AI companions. Minors on Character.AI are already pointed to a more limited version of the service, which uses a different AI model and includes restrictions such as limiting the types of characters they can access and types of conversations they can carry out.

With the new restrictions, the company plans to start using new technology to improve how it verifies users’ ages as it pushes them toward either an adult or under-age-18 experiences. To do this, the startup built a new model, and will start using software from identity-verification startup Persona, the company said.

Character.AI, OpenAI and other leading AI companies are increasingly being scrutinized for the ways their chatbots may be impacting users’ mental health. Since 2024, a number of lawsuits have been filed by parents against companies offering AI chatbots, alleging that extensive use of the technology led to a range of harms to kids, including manipulating and isolating them and leading to death by suicide.

Regulators are also engaged on the issue. The Federal Trade Commission launched an investigation into companies including Character.AI, OpenAI, Alphabet Inc.’s Google, Meta Platforms Inc., Snap Inc. and Elon Musk’s xAI over potential harms their chatbots pose to children. California recently signed into law a rule restricting the ways that chatbots can act toward users. And new bipartisan legislation introduced in the US Senate on Tuesday would block artificial intelligence companies from providing chatbot companions to minors.

In an interview Tuesday, Character.AI Chief Executive Officer Karandeep Anand said the company sees the future of AI entertainment as much more than just chat-based conversations, and the decision to stop minors from using key features is something the company would have eventually done anyway.

“I think what we are learning as an industry, given how fast it’s evolving, is that there are definitely unknown aspects of what long-term, open-ended chats can do,” Anand said. “We don’t know whether they’re good or bad. There’s definitely a lot of debate in the community around this, both in the research community as well as the user community.”

Anand said the company will keep offering a number of features to kids using the service, such as letting them create AI-generated short videos and stories that include Character.AI chatbots.

“The reason why users come to Character is they want a creative outlet. They want to be able to role play,” Anand said. “If you’ve created these vampire characters or Halloween characters, we want to continue engaging with them, but we feel that instead of just doing it over a chatbot, there are other ways to do it.”

Last year, Google hired several key members of Character.AI’s leadership team and licensed its large language model technology in a deal that valued the startup at $2.5 billion. Character.AI continued to exist independently after the deal.

More stories like this are available on bloomberg.com