Character.AI to bar children under 18 from using its chatbots

SAN FRANCISCO – Character.AI said on Oct 29 that it would bar people under 18 from using its chatbots starting late next month, in a sweeping move to address concerns over child safety.

The rule will take effect from Nov 25, the company said, adding that over the next month the company will identify which users are minors and put time limits on their use of the app to enforce it.

Once the measure begins, those users will not be able to converse with the company’s chatbots.

“We’re making a very bold step to say for teen users, chatbots are not the way for entertainment, but there are much better ways to serve them,” Mr Karandeep Anand, Character.AI’s chief executive, said in an interview.

He said the company also planned to establish an artificial intelligence safety lab.

The moves follow mounting scrutiny over how chatbots sometimes called AI companions can affect users’ mental health.

Last year, Character.AI was sued by the family of Sewell Setzer III, a 14-year-old in Florida who killed himself after constantly texting and conversing with one of Character.AI’s chatbots.

His family accused the company of being responsible for his death.

The case became a lightning rod for how people can develop emotional attachments to chatbots, with potentially dangerous results.

Character.AI has since faced other lawsuits over child safety.

Character.AI allows people to create and share their own AI characters, such as custom anime avatars, and it markets the app as AI entertainment.

Some personas can be designed to simulate girlfriends, boyfriends or other intimate relationships.

Users pay a monthly subscription fee, starting at about US$8 (S$10.40), to chat with the companions.

Until its recent concern about underage users, Character.AI did not verify ages when people signed up.

Character.AI has about 20 million monthly users, with less than 10 per cent of them self-reporting as being under the age of 18, Mr Anand said.

Under Character.AI’s new policies, the company will immediately place a two-hour daily limit on users under the age of 18.

Starting Nov 25, those users cannot create or talk to chatbots, but can still read previous conversations.

They can also generate AI videos and images through a structured menu of prompts, within certain safety limits, he said. NYTIMES