An AI Chatbot Did Better Than Group Therapy in One Student Trial. Here's What That Does and Doesn't Mean
A 995-student trial found an AI platform improved some outcomes more than group therapy, but the effects were small. How to read it, and how to vet a mental health chatbot.
Whenever a headline says a chatbot "beat" therapy, I read the numbers before I read the claim. Mental health care has a quiet problem that sits underneath all of this: there are more people who could use some support than there are people available to give it, and a waiting list is its own kind of silence. So a trial that tests an AI conversation against real alternatives deserves a careful look, neither a cheer nor a shrug.
Two randomized trials are worth reading side by side. Neither says what the loudest summaries say, and both say something useful.
What the Israeli trial measured
The larger trial, reported in JAMA Network Open, ran in Israel from April to October 2025. It enrolled 995 university students aged 18 to 35 who reported current emotional distress. They were randomly assigned, in equal thirds, to a 12-week AI conversational platform called Kai, to face-to-face group therapy, or to a waiting list.
The researchers tracked anxiety, depression, post-traumatic stress symptoms, life satisfaction and well-being. Coverage of the results says that students using the AI platform improved modestly but by a statistically greater amount than both the group therapy and waiting-list groups on anxiety, life satisfaction and well-being, and that the differences were still there at a three-month follow-up. Post-traumatic stress symptoms did not differ significantly between groups.
On depression, the summaries I found disagree. Some say the AI group improved more than the waiting list but not more than group therapy. Others report better depression outcomes than both. I cannot settle that from secondary coverage, so if depression matters to you, the paper itself is the place to check.
How big the effects actually were
This is where "beat" starts to look generous. In the AI arm, average anxiety scores on the GAD-7 moved from about 7.3 to 6.3, and depression scores on the PHQ-9 moved from about 7.6 to 6.7. Coverage notes that changes of that size fall below the amounts commonly cited as clinically meaningful. A result can be statistically real and still small for an individual student.
Two more limits belong in the picture. The outcomes were self-reported, not rated by a clinician. And attrition was substantial: around 35 percent of participants in each group were lost by the follow-up. People who stay in a study are not always the same as the people who leave, so the follow-up numbers should be read with some caution.
Why the comparison was group therapy, and what that may explain
The trial pitted the AI platform against group therapy, not against one-to-one therapy. That matters. What follows is my interpretation, not something the trial tested.
In a group, each person gets a slice of the therapist's attention. A student talking to an AI platform gets a conversation that is entirely about them, at any hour, at their own pace, without waiting for a turn. For some kinds of distress, that availability may count for a lot. The group arm, meanwhile, offers something the AI cannot, which is the experience of being heard by other people in the same room. The outcome measures here may not have captured that well.
None of this tells us how an AI platform would compare with an individual therapist. The trial did not ask that question.
The Dartmouth Therabot trial
The second trial, from Dartmouth and published in NEJM AI in March 2025, enrolled 210 adults with symptoms of depression or generalized anxiety, or at high risk of an eating disorder. Roughly half were randomly given access to a generative AI therapy chatbot called Therabot for several weeks. The rest, the control group, had no access.
The reported reductions were large on paper: about 51 percent in depression symptoms, about 31 percent in anxiety symptoms, and about 19 percent in concerns about body image and weight. These were also self-reported, and the comparison group received nothing, which makes it easier for an intervention to look good than a comparison with another treatment would. The lead author, Michael Heinz, said that no generative AI agent is ready to operate fully autonomously in mental health.
The access-gap argument
The strongest case for these tools has little to do with beating anyone. It is that many people with mild to moderate distress get nothing at all, because of cost, distance, waiting lists, or because they are not yet ready to sit across from a person. A tool that is available at two in the morning and costs little is competing against the option of no help, not against an ideal therapist.
That framing also marks the limit. These trials involved students and adults with symptoms, not people in crisis. Nothing in them suggests a chatbot is a safe stand-in for care when someone is at risk.
What clinicians still worry about
Psychologists quoted in the coverage remain skeptical, and the concerns are reasonable. Self-reported outcomes and high dropout limit what we can conclude. Follow-up was only a few months. A chatbot must know when a conversation has moved beyond what it can handle, and hand that person to a human quickly. And the data people share in a vulnerable moment is sensitive, so what a company does with it matters as much as what the bot says.
How to vet a mental health chatbot before you use one
Here is a short checklist. If an app cannot answer these plainly, treat that as an answer.
| Question | A good answer looks like | A warning sign |
|---|---|---|
| Evidence basis | It points to a specific trial or published study of this tool, with plain results | Only testimonials, or evidence about a different product |
| Scope of use | It says what it is for (everyday stress, low mood) and what it is not for | It presents itself as a replacement for therapy or medical care |
| Escalation path | It recognises crisis language and points to human help such as a hotline | No stated plan for emergencies, or it keeps chatting through a crisis |
| Data privacy | A readable policy on what is stored, who sees it, and how to delete it | Vague wording, or conversations used for advertising or sold on |
| Human oversight | Clinicians helped design it and someone reviews its safety | No named people or qualifications behind it |
FAQ
Does this mean an AI chatbot is as good as therapy?
No. The Israeli trial compared an AI platform with group therapy and a waiting list, and found modest gains in some outcomes. It did not compare it with one-to-one therapy, and the effect sizes were small.
Is it safe to use a chatbot if I am struggling?
For everyday stress or low mood, some people find them a useful supplement. They are not suitable for a crisis. If you are in danger or thinking about harming yourself, contact local emergency services, or in the United States call or text 988.
Why did the AI do better against group therapy?
The trial did not test the reason. One plausible explanation is that a one-to-one conversation on demand gives more individual attention than a share of a group session, but that is interpretation, not a finding.
What should I look for first in an app?
Look for evidence about that specific tool, a clear emergency plan, and a privacy policy you can understand. Those three catch most of the problems.
Is this medical advice?
No. It is a reading of two studies. If you are worried about your mental health, a qualified professional is the right person to talk to, and a chatbot at best sits alongside that.