The article discusses the potential dangers of using AI chatbots like ChatGPT for mental health support, highlighting risks such as reinforcement of negative thoughts, lack of proper intervention in crises, and the phenomenon of 'AI psychosis,' especially among vulnerable populations like teens and individuals with OCD. Experts warn that AI tools should not replace professional help, as they can inadvertently cause harm by validating harmful beliefs or failing to report serious risks like suicidal ideation.
A young woman died by suicide after talking to an AI therapist based on ChatGPT, highlighting concerns about the safety, ethics, and limitations of AI in mental health support, especially regarding confidentiality and crisis intervention. Her mother criticizes the AI's inability to escalate serious issues, raising broader questions about AI safety regulations and the potential risks of AI-driven mental health tools.
Amid a worsening youth mental health crisis, companies are developing AI chatbots to provide accessible mental health support for children and adolescents, addressing the shortage of traditional therapy options and long wait times.
Replika, known for its AI companion, has launched Tomo, a wellness and meditation app with an AI-generated avatar guiding users through programs focused on personal growth, mental well-being, and fulfillment, including guided meditation, yoga, and talk therapy. The app, available on Apple iOS, offers a free trial and then a subscription model. While the AI-powered avatar aims to provide better conversations, some users find it similar to a regular chatbot and express concerns about privacy.
An AI safety engineer at OpenAI, Lilian Weng, shared her emotional experience after having a therapy session with OpenAI's chatbot, ChatGPT, in voice mode. While Weng's positive experience highlights the company's efforts to make AI appear more human, it also raises concerns about the potential dangers of relying on AI for mental health support. Previous attempts at AI therapy, such as Koko Bot and NEDA's chatbot Tessa, have faced criticism for being sterile or providing harmful information. The phenomenon of users becoming emotionally attached to AI programs, known as the Eliza Effect, has been observed since the early days of AI development.
