According to recent data on data awaiting examination and publication (Rousmaniere et al. 2025) but widely disseminated, 48.7% of the 499 American respondents declared using the LLMS (large language models) for psychological support in the previous year. The majority did this to anxietypersonal advice, and depression.
They notably pointed out mainly neutral or positive responses, and 37.8% said it was better than “traditional therapy”. Only 9% said it was harmful. As a parent and clinician occupied in mental health, I saw both terrible negative and good results. Although generally an ai optimisticI see that there are clear and present dangers that we do collectively. At the same time, there is a shortage of mental health services worldwide, and AI is promising to fill the gap, if not responsible, with human surveillance and meticulous guidelines.
In the rabbit burrow
There is an increasingly alarm among accomplished mental health leaders, psychiatrists and psychotherapists, expressed on social networks. They bring in many cases of LLMS that lead people narcissistic Fugue and even psychotic declares in a sort of cyber madness for two, the official DSM-5 whose name is a “shared psychotic disorder”.
The main concern is that LLMS tell people what they want to hear, a digital accelerated version of the risks posed by therapists and ill -trained coaches, which can simply validate people’s concerns without engaging them effective psychotherapy. This phenomenon was explored on social networks with a documented risk1 posed by various “therapists” who do not follow the guidelines accepted for the use of social media2So reinforcing negative personality features. There is the additional dangerous tendency to LLM to “hallucinate”, according to how they are formed and settled.
Such use of LLMS is an unprecedented social experience, similar to powerful drugs sold over -the -counter without clinical supervision. If prescription opioid pain drugs could be purchased by anyone, the consequences include an increase in rates of dependence and an overdose. And although many LLM platforms have introduced various railings to prevent bad players, there are practically no guarantees around LLM as therapeutic substitutes.
Direct access to what is equivalent to unregulated almost therapy is abnormal, because the FDA has a range of directives on the use of digital therapy on prescription on prescription3. In addition, the FDA has specific directives for the use of digital health technologies (DHTS) in the development of drugs4 To ensure safety and for digital therapy (DTX), including virtual reality, therapeutic games such as for ADHDand applications on websites and smartphones, which all require prescription by approved clinicians (Phan et al., 2023).
Lack of regulatory consensus and surveillance
Why are LLMs not regulated for this use, but processed instead as over-the-counter supplements, which often have risks and advantages not sought after despite wild allegations to treat a variety of conditions? It is left to the small characters to reveal: “These statements have not been evaluated by the Food and Drug Administration”. Supplements can be falsified with toxins due to poor quality control, interact with prescription drugs and have serious side effects.
Several organizations, including the World Health Organization, the United States Food and Drug Administration5 and the academic groups (for example Meskó and Topol, 2023; NGO et al., 2024; Lawrence et al., 2024; Stage et al., 2024) – have published warning declarations on the non -supervised therapeutic use of Chatbot LLMS and AI in general.
Although it is always wild, far west and there is no consensus, recommendations for sure use generally include: human supervision, validation and real tests, ethical and fair design, transparency and explanation, confidentiality and data security, continuous monitoring and quality control, and research and interdisciplinary collaboration.
For mental health applications, a good starting point is the Policy of Scale responsible for Anthropic, a model to develop and adapt.
Artificial intelligence safety levels (ASL) for mental health
Anthropic, the manufacturer of Claude, was founded in 2021 by the brothers and sisters Dario Amodei (CEO) and Daniela Amodei (president), who left Openai because of concerns for the security and the management of the development of the AI. They created the manager of responsible scaling6, 7.
A large framework for ASL in mental health (ASL-MH) would extend safety levels and focus on specific use of mental health. You will find below a preliminary model that I made.
- ASL-MH 1: No clinical relevance. AI for general use without mental health functionality. Standard AI assistance for daily tasks with basic AI ethics Guidelines and no mental health restrictions.
- ASL-MH 2: Information use only. Mental health applications providing educational content and resources. Increases mental health literacy but risks disinformation and dependence. Requires medical warnings and an expert review, without any authorized personalized advice.
- ASL-MH 3: Support interaction tools. Therapy applications offering conversational support, mood monitoring and crisis connections. Provides emotional support 24/7 but risks users who confuse AI with therapy and high -risk cases. Requires human surveillance and is prohibited in high acuity contexts.
- ASL-MH 4: Clinical auxiliary systems. Systems providing support for clinical decision and structured assessments. Improves the accuracy of the diagnosis but the risks bias and too much of a lot. Limited to approved professionals with clinical validation and transparent algorithms required.
- ASL-MH 5: Autonomous mental health agents. AI provides personalized therapeutic advice. Offers evolving and personalized treatment but risks psychological dependence and manipulation. Requires co-managed care with compulsory human surveillance and restricted autonomy.
- ASL-MH 6: Experimental superintendent area. Advanced therapeutic reasoning systems with unknown capabilities. Potential for revolutionary treatments but has risks of emerging behavior and massive influence. Limited to research environments with international monitoring and deployment moratories.
Future directions
The following steps would include the formation of a panel of experts from experts from the main stakeholders in the government, private and public sectors, representatives of citizens, automatic learning leaders and academic and clinical experts in mental health.
This is a call for action. Although the proverbial horse is out of the barn, it is not too late for the main stakeholders involved in the development and use of these very precious but easily poorly used tools to adopt universal standards. Government regulation agencies, such as DTX prescription applications currently available, are responsible for supervising the useless use of LLM and even more sophisticated AI technologies.