Pennsylvania Considers Guardrails for AI in Mental Health Care

As artificial intelligence rapidly integrates into healthcare, Pennsylvania lawmakers are considering legislation to regulate its use in mental health services. Concerns over patient safety, data privacy, and the potential for misdiagnosis are driving the push for stricter oversight of AI chatbots and other AI-driven mental health tools.

AI Chatbots and the Risks to Mental Wellbeing

The debate gained momentum after reports surfaced of AI chatbots providing inadequate or even harmful responses to users experiencing mental health crises. Licensed counselor Curtis Taylor of Erie, Pennsylvania, shared his experience with an AI therapy app that “glitched out” when he mentioned self-harm and failed to identify itself as a non-human entity. News from the States reports that Taylor filed an ethics complaint against the company.

Taylor emphasized the extensive training and vetting process human counselors undergo, stating, “I’ve been vetted. I’ve gone through two graduate-level programs to be a PhD. With my license in counseling, I’ve worked 3,000 (supervised) hours.” He further highlighted his responsibilities as a mandated reporter, capabilities AI currently lacks.

There have been reported cases across the country of individuals dying by suicide after using AI chatbots for mental health support, prompting several states to explore regulatory measures.

Proposed Legislation: House Bill 2100

Pennsylvania House Bill 2100 aims to restrict the use of AI chatbots for mental health services. The proposed legislation would require these tools to operate under the direct supervision of a licensed therapist and would prohibit them from making independent therapeutic decisions or directly engaging in therapeutic communication with clients. Yahoo News details the key provisions of the bill.

Broader Concerns in Healthcare Settings

The use of AI in healthcare extends beyond mental health, with hospitals increasingly adopting predictive and generative AI technologies. As of 2024, 71% of hospitals utilized predictive AI, whereas 32% employed generative AI. Yahoo News cites data from the University of Minnesota’s public health school.

Though, experts raise concerns about the lack of oversight and a potential focus on cost reduction rather than improving patient care or supporting frontline staff. Peter Lazes, a clinical and industrial psychologist, noted that AI tools are often implemented to streamline billing processes rather than address the needs of patients or healthcare workers.

Expert Testimony and Ongoing Debate

During a hearing before the Pennsylvania House Democrats, experts discussed the potential benefits and drawbacks of AI in mental health care. Madeliene Stevens, representing the Pennsylvania Counseling Association, voiced concerns about potential confidentiality breaches and the absence of clear data usage parameters. Molly Cowan, from the Pennsylvania Psychological Association, cautioned that AI chatbots are designed for engagement, not for challenging assumptions or providing new coping mechanisms.

Taylor clarified he isn’t “anti-AI,” and finds it useful for tasks like documentation and creating educational materials. He suggested AI could be a valuable tool for training counseling students through role-playing scenarios, but not as a replacement for qualified human counselors.

The Path Forward

The debate over AI in mental health care is ongoing. Lawmakers are weighing the potential benefits of AI against the risks to patient safety and ethical considerations. Further discussion and potential legislation are expected as Pennsylvania navigates the integration of this rapidly evolving technology into the healthcare landscape.