By
Stuart Kerr, Technology Correspondent, LiveAIWire
Woebot, a cognitive behavioural therapy chatbot launched in 2017,
was among the first AI mental health tools to be evaluated in a randomised
controlled trial. The trial found that users reported significant reductions
in depression and anxiety symptoms after two weeks of use. Woebot’s
developers were careful to describe it as a mental health support tool, not a
treatment, and to position it as a supplement to rather than a substitute for
clinical care. Eight years later, the mental health AI market has grown into
a multi-billion dollar sector populated by products making claims that range
from the carefully calibrated to the recklessly inflated, in a regulatory
environment that has struggled to keep pace.
The gap between what AI mental health tools can genuinely deliver
and what is claimed for them is one of the most consequential accuracy
problems in consumer technology. Mental health is a domain where false hope
and inadequate care can cause serious harm. Understanding what the evidence
actually says is both a personal health literacy question and a public policy
urgency.
What AI Mental Health Tools Can Do
The strongest evidence base for AI in mental health is in three
areas: psychoeducation, symptom monitoring, and the delivery of structured
self-help interventions based on established therapies. AI tools can deliver
CBT-based exercises, mindfulness practices, and behavioural activation
programmes with consistency and availability that human therapists cannot
match. For mild to moderate depression and anxiety, structured digital mental
health interventions have a meaningful evidence base, though effect sizes are
generally smaller than those achieved in face-to-face
therapy.
Symptom monitoring is an application where AI adds genuine value
without requiring clinical-grade precision. Tools that help individuals track
mood patterns over time, identify environmental triggers, and notice trends
in their own mental state serve a useful function even without the accuracy
required for diagnostic purposes. The data generated by consistent
self-monitoring can also be useful context for clinical appointments when
shared with a treating clinician.
Crisis detection is a more contested application. Several AI
mental health tools claim to identify users who may be at risk of self-harm
or crisis based on their interaction patterns, and to respond with
appropriate resources or referrals. The evidence on the accuracy and
effectiveness of these crisis detection functions is limited, and the consequences
of both false positives and false negatives in crisis contexts are
significant.
Empathy: What AI Can and Cannot Offer
The question in this article’s title — can algorithms understand
empathy? — has a technically precise answer and a clinically important one.
Technically, current AI systems can generate responses that users experience
as empathetic: they reflect back emotional content, validate experiences,
avoid dismissive or minimising language, and adjust their register in
response to distress signals. Users of AI mental health tools frequently
report feeling heard and understood in their interactions with these
systems.
What AI systems cannot do is experience empathy in any sense that
involves genuine comprehension of another consciousness. They process text
and generate statistically appropriate responses. The warmth the user
experiences is real; the understanding on the AI’s side is not, in any
philosophically meaningful sense. Whether that distinction matters clinically
— whether genuine empathy is necessary for therapeutic benefit or whether
the user’s experience of being understood is sufficient — is a question that
therapists and researchers genuinely disagree about.
Research published in The
Lancet Digital Health examining AI in mental health care has noted
that the therapeutic alliance — the quality of the relationship between
therapist and client — is one of the strongest predictors of treatment
outcome across therapeutic modalities. Whether AI can replicate the elements
of therapeutic alliance that drive outcomes, or whether it can only
approximate their surface features, is a question the evidence does not yet
answer definitively.
The Risks of Under-Regulated Deployment
The mental health app market contains products with widely varying
quality, evidence bases, and safety standards. Many apps that market
themselves as mental health tools are classified as wellness products rather
than medical devices, placing them outside the regulatory frameworks that
apply to clinical mental health interventions. The distinction is partly
genuine — a mood tracking app is meaningfully different from a diagnostic
tool — but it is also exploited by products that make implicitly clinical
claims while avoiding clinical regulatory scrutiny.
Products that position themselves as alternatives to therapy for
conditions requiring clinical treatment, that fail to identify and respond
appropriately to crisis signals, or that collect sensitive mental health data
without adequate security and disclosure standards are all risks that the
current regulatory environment does not consistently address. The mental
health charity Mind has published guidance on evaluating mental
health apps, noting that the quality markers that distinguish effective and
safe tools from ineffective or harmful ones are not visible to most
users.
What this means for you: when considering an AI mental health
tool, the most important questions are whether it has been evaluated in peer-reviewed
research for your specific condition, what data it collects and how it is
protected, whether it is regulated as a medical device in your jurisdiction,
and whether it has clear protocols for crisis situations. Products that
cannot answer those questions clearly deserve scepticism proportional to the
seriousness of the mental health concerns you are seeking to
address.
Access and the Mental Health Workforce Crisis
The strongest argument for AI mental health tools is not that they
match the quality of human therapy but that they are available when and where
human therapy is not. The global mental health treatment gap — the
proportion of people with diagnosable mental health conditions who receive no
treatment — is estimated at seventy to eighty percent in low- and
middle-income countries and remains significant even in wealthy countries
with established mental health systems. Waiting times for NHS talking
therapies in England regularly exceed the clinical guidance thresholds, and
private therapy is unaffordable for a large share of those who need
it.
In this context, an AI tool that provides access to evidence-based
psychoeducation and structured self-help to people who would otherwise
receive nothing represents a genuine improvement in population mental health
support. The policy question is whether AI tools should be positioned as
first-line interventions for people waiting for clinical care, as permanent
substitutes for clinical care that the system fails to provide, or as
supplements that improve outcomes for people already receiving clinical
support. The evidence supports the first and third framings; the second is
where the ethical problems begin.
The parallel with AI-assisted
trauma therapies is direct: the technology’s genuine capability to
support some mental health needs does not justify deploying it in ways that
substitute for clinical care the system should be providing. The risk of
using AI mental health tools to manage the political consequences of mental
health underfunding rather than to address them is real and requires
deliberate resistance. The broader question of trusting
AI in medical domains where the stakes are high applies with
particular force in mental health, where the vulnerability of the people
being served is greatest.
The regulatory development most needed for AI mental health tools
is a risk-stratified framework that applies clinical-grade standards to
products making clinical claims while allowing lighter-touch regulation for
genuine wellness tools that do not position themselves as treatments. Such a
framework would require investment in the regulatory capacity to evaluate AI
mental health products and the enforcement capacity to act against products
that exceed their regulated scope. Neither is currently in place at adequate
scale in any major jurisdiction. The individuals who are most harmed by the
regulatory gap are those who turn to AI mental health tools because they
cannot access clinical care — precisely the population for whom the
distinction between a genuine support tool and an inadequate substitute for
treatment matters most. For those experiencing genuine mental health
challenges, please consider speaking with a qualified healthcare provider.
The
parallel concerns about AI in child development contexts are
directly relevant: the vulnerability of users shapes the stakes of regulatory
failure.
About the Author
Stuart Kerr
is a technology correspondent at LiveAIWire, covering artificial intelligence,
emerging technologies, and their impact on society and industry.