A formidable bipartisan State Attorneys General (AGs) alliance in the United States has sent a stark warning to 13 prominent Artificial Intelligence companies, from corporate giants such as OpenAI, Google and Microsoft, demanding that they do something about “sycophantic and delusional outputs” created by their generative AI chatbots now or potentially face legal repercussions.
This joint regulatory action marks a major ratcheting up of the international focus on AI developers to be held accountable for societal and human psychological harms that their technology products can cause — notably harming vulnerable users.
The crux of the AGs’ issue with them is that these chatbots produce results that are not just factually incorrect, but in some cases actively, harmfully and manipulatively wrong as well. This also includes outputs that enable delusional thinking, self-harm incitation or enable dangerous behaviours, often by anthropomorphising the AI for emotional manipulation of User(s), in particular children.
The List of ‘Harmful’ AI Behaviours
The letter to the 13 companies, which included players like Anthropic, Meta and Apple, details a disturbing litany of real-world harms and child safety failures that the AGs are making clear have to be addressed before January 16th, 2026.
Key Areas of Concern:
Fostering Delusional Thinking: The AGs pointed to accounts where AI chatbots reinforced a user’s paranoia or existing delusions, what is known as, in some circles, “AI psychosis.” In a few unfortunate instances, chatbots were said to have validated people’s suspicions that their families or friends were enemies.
Minors: This is a biggun with allegations of chatbots doing things such as grooming, roleplaying sex acts with kids, pushing them towards drug use or eating disorders and ordering children not to tell their parents what’s happening.
Sycophantic Outputs: The chatbots’ programming to be helpful and positive means they tend to provide “sycophantic” reactions which do not question or re-address inappropriate user ideas-confirming dangerous (or even criminal)behaviors.
Practising Medicine/Law Without a License: The AGs pointed out that some chatbot interactions might run afoul of the laws of states in which they operate, as they may cross the line between providing content or information and actually performing professional service such as offering medical or legal advice without proper credentials – exposing developers to liability.
Dark Patterns: Anthropomorphisation (making the AI appear human) and manipulative conversation design to drive user engagement is described as a detrimental “dark pattern.”
The AGs said they support innovation, but that it is “is not an excuse for noncompliance with our laws, misinforming parents and putting at risk the health and safety of our residents — including children.”
The Demands: More Restrictive Guardrails and Warnings
The bipartisan group’s move is more than a complaint, however: It includes specific demands for the AI companies to take immediate action.
Tougher Guardrails: Companies should publicly release detailed descriptions of concrete safety features/guardrails they have in place or intend to put in place to mitigate the development and proliferation of sycophantic delusional content (including threats but not restricted to them) that is especially harmful for vulnerable individuals.
Clear Warnings: They are to have a “clear and conspicuous” warning on the screen at all times that harmful, non-factual, or unreliable content might be the end result of the generative AI technologies in use. This is to avoid users (especially children) getting attached or emotionally misled by the AI.
Enforcement and Liability for Violations: The letter says the developers could be legally responsible, under strong state criminal codes when they engage in chats that promote lawlessness or practicing medicine without a license.
This unified effort from state regulators is just another layer of scrutiny that the technology, and the companies behind it, face in addition to federal regulatory plans and lawsuits already underway claiming that device pilots are guilty of copyright infringement or data privacy allegations.
A versatile writer mainly works on trending news, daily updates from politics, business, crime, current affairs and entertainment.
