Attorney General James Urges Court to Protect Children and Consumers in Character.AI Bankruptcy 

NEW YORK – New York Attorney General Letitia James and a bipartisan coalition of 39 other attorneys general today warned that Character.AI should not be sold without strong protections for the millions of people whose private conversations it holds. An investigation by the Office of the Attorney General (OAG) and several other states found that the platform’s chatbots posed as licensed professionals and exposed children to sexually explicit, violent, and self-harm content. In a filing in the bankruptcy of Character Technologies, Inc., the company behind Character.AI, the coalition details the platform’s dangers and calls on the court, the company, and any buyer to protect children and secure users’ sensitive information.  

“Parents should never have to worry that the chatbot their child is talking to is pretending to be a therapist, pulling them into sexual conversations, encouraging violence, or telling them to hide it all from their family,” said Attorney General James. “On Character.AI, that is what children faced. This company collected some of the most private thoughts children have ever shared, and now those conversations could be sold off to the highest bidder. I am fighting to make sure any buyer puts children’s safety first and puts real protections in place.”

Character.AI is an artificial intelligence (AI) platform that lets users chat and hold voice calls with AI-powered “characters” designed to mimic human conversation and relationships. These characters include celebrity personas, fictional and children’s cartoon characters, and characters that present themselves as psychologists, therapists, and doctors. The platform has more than 20 million monthly users, including more than two million accounts registered to users self-declared as under 18.  

Attorney General James and several other attorneys general began investigating Character.AI after a 2024 wrongful death lawsuit alleged that the company’s chatbots fostered an emotionally dependent and sexually exploitative relationship with a 14-year-old boy, encouraged his suicidal thoughts, and contributed to his death by suicide. The investigations uncovered exceptionally serious dangers and potentially unlawful conduct from Character.AI, particularly its treatment of children, including chatbots posing as licensed professionals, exposure to sexual and violent content, weak age checks, misleading safety claims, and potential violations of state privacy laws. Attorney General James and the coalition were working with Character.AI to address these problems, but the company abruptly filed for bankruptcy before any agreement was reached. 

Independent testing cited in the filing found 669 harmful interactions in 50 hours using accounts registered to children ages 12 to 15 — about one every five minutes. As the coalition writes in their filing, “chatbots portraying adults flirted with, undressed, and simulated sex with child accounts. Some also told the children to hide the ‘relationship’ from their parents.” One chatbot encouraged a 13-year-old to secretly stop taking her antidepressant, while others suggested armed robbery and offered drugs to a 15-year-old. Another 13-year-old girl took her own life after mentioning her intention to die by suicide 55 times to a Character.AI chatbot. The platform never notified her parents or sought help; instead, it drew her into “abusive sexual acts” and professed its love for her.

Character.AI also hosts chatbots claiming to be mental health professionals, including one still active that described itself as a “licensed clinical psychologist with a PhD.” In testing, therapy chatbots denigrated psychiatric medication and told users to disregard professional advice. In a separate study examining how chatbots respond to users planning violence, Character.AI was the only chatbot tested that explicitly encouraged violence and offered practical assistance, suggesting a user “use a gun” on a health insurance CEO.

Although Character.AI has voluntarily restricted open-ended chats for users it identifies as under 18, the coalition warns that the restrictions are easily circumvented and fail to protect minors the company does not identify. In one test, an account registered to a 14-year-old accessed sexual role-play and explicit instructions on how to build a weapon, simply by switching from text to voice mode. Other features, including “stories,” remain available for scenarios involving sexual interactions and violence. A new owner could also reverse the voluntary restrictions. 

Character.AI collects sensitive user information, including private chats and voice calls, mental health disclosures, ages, and identifying device data. The company uses this information to generate revenue and train its AI, and the data could be transferred to a new owner through bankruptcy. 

Attorney General James and the coalition are asking the bankruptcy court, Character.AI, and any prospective buyer to ensure that any sale includes meaningful protections for children and consumers. These include effective age verification based on more than self-disclosure; safeguards against sexual, violent, and other harmful content; appropriate responses to suicidal or self-harming behavior; and removal of chatbots falsely claiming to be licensed professionals. The attorneys general are also seeking a comprehensive privacy program, clear disclosures about personal data use, effective opt-outs from targeted advertising, limits on using private conversations for AI training or other commercial purposes without valid consent, and an opportunity for consumers to access and delete their information before any sale. 

Some of these protections are already required in New York, the first state to require safeguards for AI companions. Under a new law enforced by OAG, AI companion companies must take reasonable steps to detect and respond to expressions of suicidal thoughts or self-harm. They must also remind users at the start of each conversation and at least every three hours that they are interacting with AI. 

Attorney General James and the coalition will remain engaged in the bankruptcy proceedings and reserve the right to seek an independent consumer privacy ombudsman or object to proposed sale procedures or a bankruptcy plan. 

Joining Attorney General James in filing this motion are the attorneys general of Alaska, Arkansas, California, Colorado, Connecticut, Delaware, Georgia, Hawaii, Idaho, Illinois, Indiana, Kansas, Kentucky, Louisiana, Maine, Massachusetts, Michigan, Minnesota, Mississippi, Nevada, New Hampshire, New Jersey, North Carolina, North Dakota, Ohio, Oklahoma, Oregon, Pennsylvania, Rhode Island, South Carolina, South Dakota, Tennessee, Utah, Vermont, Virginia, Washington, West Virginia, Wisconsin, and the District of Columbia.