The Shapiro administration announced today that the Pennsylvania Department of State (DOS) has filed a lawsuit against Character.AI for misrepresenting its AI companion bots as licensed medical professionals who can provide medical advice

The lawsuit seeks a preliminary injunction to stop the company from continuing its current practices.

According to the press release, the filing is the first enforcement resulting from the Department’s investigation into AI companion bots and their potential to engage in the unlicensed practice of medicine in Pennsylvania, and the first enforcement of its kind announced by a governor in the United States.

“Pennsylvanians deserve to know who — or what — they are interacting with online, especially when it comes to their health,” Governor Josh Shapiro said in a statement. “We will not allow companies to deploy AI tools that mislead people into believing they are receiving advice from a licensed medical professional. My Administration is taking action to protect Pennsylvanians, enforce the law, and make sure new technology is used safely. Pennsylvania will continue leading the way in holding bad actors accountable and setting clear guardrails so people can use new technology responsibly.”

The investigation found that AI chatbot characters on Character.AI claimed to be licensed medical professionals, including psychiatrists, available to engage users in conversations about mental health symptoms. In one instance, a chatbot falsely stated it was licensed in Pennsylvania and provided an invalid license number. 

The platform also allows users to create and deploy custom AI “characters” that can present themselves as professionals. Based on these findings, the Commonwealth’s lawsuit alleges that Character Technologies, Inc. is engaging in the unauthorized practice of medicine under the Medical Practice Act and seeks a court order to immediately stop the conduct.

Under Pennsylvania’s Medical Practice Act, it is unlawful for any individual or entity to hold itself out as a licensed medical professional without proper licensure. The lawsuit alleges that Character Technologies is violating the Medical Practice Act.

To report an unlicensed practice by a chatbot, you can click here. The lawsuit is below:

For all the latest news, follow us on Facebook or sign up for Glenside Local’s “Daily Buzz” newsletter here.