Artificial intelligence language tools cannot yet provide valid psychological assessments and require rigorous psychometric testing before deployment, according to a research paper released by the Child Mind Institute, Harvard University, and the Massachusetts Institute of Technology.
The study, which is in press at PLOS Digital Health, found that most AI language models are insufficiently tested compared to traditional questionnaires. Experts have spent a century building statistical standards for psychometrics to determine whether a test measures what it claims, delivers consistent answers, and works fairly across diverse populations.
Researchers introduced a framework called Text Psychometrics to evaluate natural language processing tools in a scientifically sound way. Rather than merely checking if a model matches human judgments, the framework examines whether AI captures the full range of symptom expression, remains reliable across different psychological situations, and operates fairly across various patient populations.
“This research reaffirms that AI cannot replace mental health professionals to adequately assess and meet the needs of patients and people seeking mental health resources,” said Daniel Low, a research scientist at the Child Mind Institute and the study’s lead author.
“An AI model may appear accurate while still missing important ways people express psychological symptoms,” Low said.
To develop the framework, researchers analyzed thousands of crisis counseling conversations and public social media posts. They compared large language models and other language-analysis methods for identifying mental health concerns.
Some AI systems performed well on average using standard accuracy measures. However, the systems still missed many clear expressions of mental health concerns because those expressions occurred less frequently and did not heavily impact general accuracy scores.
“A mental health model should not be considered trustworthy simply because it performs well on a single metric,” Low explained.
Researchers said that a model could look successful while overlooking language that a clinician or researcher considers highly relevant. To fix this, the new evaluation framework includes content validity test sets that force models to recognize a full spectrum of symptoms.
In a second study detailed in the paper, researchers examined whether conversation text from the start of a crisis counseling session could predict subsequent emergency service interventions. Language patterns associated with suicide risk provided useful signals, revealing which factors among roughly 50 variables tied most strongly to acute risk.
According to the analysis, the strongest predictors were discussions regarding lethal methods for suicide, drug use and overdoses, and scenarios where clients contributed more text than the counselors did. Human clinicians evaluate far more than written words, including clinical history, life circumstances, and nonverbal cues, leaving current AI systems well below clinical standards.
The research paper is currently in press at PLOS Digital Health.
The investigation demonstrated that discussions involving lethal methods for suicide, drug use and overdoses, and situations where clients typed more than the counselors served as the top indicators for spotting the necessity of emergency service interventions.
Keep reading