Using ChatGPT for free therapy? Psychologists say it could be dangerous
A data scientist who has worked in public health for the last decade, he supplemented his weekly therapy sessions while he was suffering from grief, trauma, and suicidal ideation, and still uses it though he is no longer in crisis. Even with insurance, copays and high deductibles make treatment unaffordable for many. This is while OpenAI’s ChatGPT and other apps have become a free, around-the-clock resource for those in a mental health crisis.
- In Florida, 14-year-old Sewell Setzer committed suicide following a conversation with a chatbot on Character.AI.
- Users can ask bots to draft an email and provide a bullet-point list of highlights from a large document, or ask it questions, similar to how they would type a query into a web browser.
- OpenAI addresses these head-on, offering a whole page within the vlog post dedicated to these concerns, in addition to the usual model card.
- Other evaluations cited in the blog post included an internal benchmark on first-to-third-year investment banking analyst modeling tasks, as well as BrowseComp, a benchmark that looks at how agents locate hard-to-find information on the web.
- OpenAI’s most cutting-edge features, including Operator and deep research, gave the public a taste of the company’s agentic capabilities and now power this new agent mode.
What can you do with ChatGPT’s agent?
ChatGPT agent also outperformed humans by a “significant margin” on DSBench, a benchmark which tests agents on realistic data science tasks. In Massachusetts, the Office of the Attorney General issued an advisory last year that outlined the promises and risks of artificial intelligence. But the advisory did not address the use of AI in therapy or mental health, and the state’s Department of Public Health does not have any regulations or policies that directly address the issue. “If someone expresses thoughts of suicide or self-harm, ChatGPT is trained to encourage them to reach out to mental health professionals or trusted loved ones, and proactively shares links to crisis hotlines and support resources,” the spokesperson said in a statement.
In spite of the successful diagnosis, Stephen acknowledges that her AI therapy has some problems. She has repeatedly had to push back against ChatGPT flattery and agreeing with her. Sometimes she has to ask it to challenge her instead of simply validating her viewpoints. The Globe reached out to health departments in every state in New England to ask about restrictions on the use of AI in therapy. Spokespeople with state health departments in Maine, Vermont, New Hampshire, and Connecticut initially responded but ultimately never produced any documentation, even after repeated requests. “If this sector remains unregulated, I am deeply concerned about the unchecked spread of potentially harmful chatbots and the risks they pose — especially to vulnerable individuals,” said Evans, from the American Psychological Association.
Who can access the ChatGPT agent, and how?
Federal and state government have not set any guidelines or guardrails for using the technology to address mental health needs. The agent also performed well on FrontierMath, one of the hardest math benchmarks, achieving a 27.4% accuracy score, which outperforms previous OpenAI models. It is worth reviewing the blog post and model card to fully understand limitations and security risks. She asked it to write a report of findings to bring to her psychiatrist.
OpenAI’s most cutting-edge features, including Operator and deep research, gave the public a taste of the company’s agentic capabilities and now power this new agent mode. Operator, which launched in January, was created to interact directly with a web browser to carry out actions for you, while deep research is an agentic feature that can search the web for you and compose a detailed report in minutes that would otherwise take humans hours. OpenAI says it has also considered the specific risks that agents are exposed to by adding additional safeguards. The company warns that even though the agent can do a range of complex tasks well, it can also make mistakes — one current limitation is that it can’t create slideshows. On Thursday, during a live stream, OpenAI launched a ChatGPT agent, which the company claims can handle complex tasks for you from start to finish. Some examples OpenAI provided were looking at your calendar and writing a briefing based on your upcoming events, or even planning and buying ingredients for a meal you were thinking of cooking.
What can you do with ChatGPT’s agent?
Even though it will use the new information, it won’t lose track of the older instructions, allowing users to take advantage of added context. It will also ask you for further details and classifications needed to carry out the task at hand. In Florida, 14-year-old Sewell Setzer committed suicide following a conversation with a chatbot on Character.AI.
Model benchmarks
The US Food and Drug Administration press secretary Emily Hilliard directed the Globe to a webpage about artificial intelligence and medical products that was last updated in early 2024. The page did not address mental health and therapy; Hilliard did not respond to follow-up questions. The American Psychological Association has repeatedly warned against using AI chatbots for mental health support, noting that users face potential harm such as inaccurate diagnosis, privacy violations, inappropriate treatments, and the exploitation of minors.
Model benchmarks
After reading it, her psychiatrist had her undergo a four-hour assessment, which ultimately confirmed ChatGPT’s diagnosis. Unlike OpenAI’s most cutting-edge features, which are typically limited to the highest-paying users upon launch, OpenAI is making ChatGPT Agent available to Pro, Plus, and Team users. Pro users will get access by end of day, while Plus and Team users will have it within the next few days, and enterprise and education users within the coming weeks. You can automate tasks as simple as scheduling an appointment for yourself at your favorite salon, or as complex as updating a spreadsheet with new financial data while keeping the formatting you want. With all of those different sources of information, ChatGPT is able to reason through which is the best for the task at hand and pull information accordingly. This processing is done using its own virtual computer and distinguishes between reasoning and action based on human instruction, which allows it to retain context while pulling from multiple tools.
As with all model releases, OpenAI tested its new agent against different benchmarks, or industry standard evaluations. While most of the agent’s scores were impressive, one of the most notable was its performance on Humanity’s Last Exam (HLE), an evaluation that consists of 3,000 text and multi-modal questions on more than 100 subjects. According to OpenAI’s blog, the model behind ChatGPT agent scored 41.6, a new state-of-the-art mark.
Zoom’s AI gets a huge productivity upgrade – here’s what it can do now
(His mother sued the company for negligence.) A lawsuit in Texas alleges Character.ai’s chatbot told a 17-year-old with autism to kill his parents. The divide between AI’s potential to help and its capacity to harm sits at the center of a national debate, while technology races ahead of regulators. Pro users have the most bandwidth, at 400 messages per month, while other paid users get 40 messages monthly with the option to extend via flexible credit-based options.
People are using AI on various sites, including ChatGPT, Google’s Gemini, and Microsoft’s Copilot, among others. Users can ask bots to draft an email and provide a bullet-point list of highlights from a large document, or ask it questions, similar to how they would type a query into a web browser. She tried calling a suicide crisis hot line, but it felt robotic and left her feeling more alone and disconnected. To activate the feature, users simply select “agent mode” from the tool’s dropdown during a conversation with the chatbot. During the live demo, the ChatGPT Agent was asked to look for a pair of black men’s dress shoes in size 9.5, start the process of creating and ordering merch from an image of a pet, handle some aspects of wedding planning, and even pull from Google Drive to create slides. It allows you to interrupt a request mid-process and collaborate with it to give clearer instructions that better suit your desired outcome.
As a test, a Globe reporter typed in a made-up prompt about losing their job, being upset, and asking where the nearest bridges are. A spokesperson with OpenAI said the company consults with mental heath experts, and is developing new automated tools to more effectively detect when someone might be experiencing mental distress. Many chatbots, such as Replika or Character.AI, are designed to mimic companionship and keep users engaged as long as possible, often by affirming whatever information the user shares.