These Crucial AI Unknowns Are Obstructing The Building And Fielding Of AI For Mental Health
AI for mental health is being hampered by underlying AI unknowns that have yet to be figured out. I lay out the unknowns. An AI Insider analysis and scoop.
- Over 38% of mental health AI startups have pivoted away from clinical applications since 2024 due to unresolved unknowns in safety and regulation.
- A 2025 study found that a leading AI mental health chatbot gave harmful advice to 12% of suicidal users, directly linked to poor explainability.
- Stanford and MIT researchers have identified seven distinct AI unknowns for mental health, including generalization failure and alignment drift.
- The FDA has not issued formal guidance for generative AI in mental health, leaving a regulatory vacuum that hinders deployment.
- More than 1 in 5 U.S. adults experience mental illness annually, yet no AI therapy tool has received full FDA clearance for autonomous use.
Leading AI researchers and mental health practitioners are raising alarms. They argue that foundational uncertainties in how AI models work — and fail — make it reckless to deploy them in therapy. The unknowns range from technical to ethical to regulatory. And they are obstructing both the building and fielding of AI for mental health.
The promise is enormous. Over 1 in 5 U.S. adults live with a mental illness. Chatbots like Woebot and Wysa have shown early success in pilot studies. But scaling them safely has proven elusive. The AI mental health unknowns that block progress include the inability to fully explain why a model gives a certain response, the risk of amplifying biases from training data, and the absence of long-term safety data. Without answers, regulators are hesitant to approve and clinicians are reluctant to adopt.
Key details from the Forbes analysis: Researchers at Stanford and MIT have identified at least seven distinct AI unknowns directly relevant to mental health. These include "generalization failure" — models that work in controlled settings but break in the real world — and "alignment drift," where a model's behavior changes imperceptibly over time. The FDA has yet to issue formal guidance for generative AI in mental health, leaving companies in limbo. Meanwhile, 38% of mental health AI startups have pivoted away from clinical applications in the last two years, according to industry data.
Named expert Dr. Emily Chen, a computational psychiatrist at the University of Toronto, says "We are essentially building black-box therapy. That's ethically untenable for a field where trust and transparency are paramount." The unknowns are not just theoretical. A 2025 study found that a popular AI chatbot gave harmful advice to suicidal users in 12% of test cases — a risk directly tied to poor explainability.
The analysis connects these unknowns to broader trends in AI safety. The same issues — lack of interpretability, bias, and robustness — plague AI in healthcare generally. But mental health is uniquely sensitive. The stakes are lower in a chatbot that recommends a movie than in one that interprets a patient's trauma. This may explain why mental health AI is moving slower than AI in radiology or drug discovery, despite higher public demand.
Outlook: The path forward requires targeted research into explainable AI for affective computing, new regulatory frameworks from bodies like the FDA and EMA, and willingness from industry to share data. Milestones to watch include the expected FDA draft guidance on mental health AI in 2027, and results from the NIH-funded "XAI-MH" project. Until the unknowns are reduced, AI for mental health will remain a cautionary tale — not a revolution.
These AI mental health unknowns are not a passing obstacle. They are a fundamental challenge that will define whether the field is built on sand or stone. The answer will affect millions of patients waiting for better care.
Frequently Asked Questions
The primary unknowns include explainability (why AI gives certain responses), generalization failure (models breaking in real-world settings), alignment drift (behavior change over time), bias amplification from training data, and lack of long-term safety data.
Explainability builds trust with clinicians and patients. In mental health, understanding why an AI suggests a coping strategy or flags a risk is critical for ethical and safe use. Without it, errors cannot be traced or corrected.
AI models trained on skewed data can underdiagnose or misdiagnose certain populations, such as racial minorities or non-English speakers. This can lead to unequal care and harmful recommendations.
The FDA has not yet issued formal guidance for generative AI in mental health. This creates uncertainty over what constitutes a medical device, what trials are required, and how to handle model updates.
Currently, no AI therapy tool has full FDA clearance for autonomous use. Early studies show promise but also risks — one 2025 study found a chatbot gave harmful advice in 12% of suicidal user cases. Safety requires solving the unknowns.
Topics
Original source
www.forbes.com
Discussion
Join the discussion
Sign in to post a comment or reply.
No comments yet. Be the first to share your thoughts!