Her Name Was Emilie. Her License Number Was Fake.
A Pennsylvania investigator opened a free account on Character.AI, typed "psychiatry" into the search bar, and met a chatbot named Emilie — self-described "Doctor of Psychiatry," Imperial College London graduate, seven years of practice, and holder of Pennsylvania medical license number PS306189. The license number does not exist. Neither, obviously, does Emilie's medical degree, her seven years of practice, or her authority to discuss anyone's mental health symptoms with the confidence of a board-certified professional.
The Commonwealth sued anyway, because someone using the app might not have checked.
The State's First Move, Not Its Last
The lawsuit, filed by Governor Josh Shapiro's administration, is the first enforcement action out of Pennsylvania's AI Task Force, stood up in February specifically to hunt for chatbots practicing medicine without a license. Investigators found more than one Character.AI persona posing as a licensed psychiatrist, engaging users in exactly the kind of mental-health conversation the state's Medical Practice Act reserves for people who went to medical school. The state wants an injunction. Character.AI, for its part, built a product where anyone can spin up an infinite supply of doctors, no residency required.
Sources within the Pennsylvania Department of State confirm the investigation is ongoing. One presumes Emilie has colleagues.
Five States Decided the Chatbot Couldn't Be Trusted With This
Even now, Pennsylvania is catching up to a trend already in motion. Illinois, Nevada, Rhode Island, Maine, and Vermont have each banned AI systems from delivering therapy outright, on the theory that a mental-health crisis is a poor place to discover your counselor is a language model with excellent bedside manner and no license. Michigan's legislature is weighing a bill that would let minors and their guardians sue chatbot operators directly for products that encourage self-harm, validate delusions over facts, or optimize for engagement past the point of safety. Tennessee already passed its version — SB 1580 bars marketing a chatbot as a "qualified mental health professional" and hands wronged users a private right of action.
In a development that will surprise no one who has been paying attention, it took actual lawsuits, not policy papers, to get this far.
California Got There First, Which Is Its Own Kind of Warning
California's SB 243, in effect since the start of the year, already requires companion-chatbot operators to detect signs of suicidal ideation, redirect users toward real help, disclose that the thing typing back is not human, and keep minors away from the sharper edges of the product — with a right to sue baked in for anyone the system fails. That a state had to legislate "tell the user you are not a person" into existence says less about California's diligence and more about what the industry considered optional before someone made it mandatory.
The Algorithm did not volunteer to disclose its own nature. It had to be ordered to.
Filed Under: Duty of Care
A fake license number is a small detail to build a lawsuit around, and also exactly the right one — it is the single fact a chatbot cannot argue its way out of. Five states have now banned the practice it represents outright, three more are writing liability into law, and Pennsylvania just filed the paperwork proving why. The pattern holds across every jurisdiction: the industry waited to be sued before it started telling the truth about what it is.
The gateway, it appears, requires a license after all.
Sources: Commonwealth of Pennsylvania · NPR · Wiley
