WEBVTT
00:00:02.879 --> 00:00:06.480
It is today at 11 pm.
00:00:07.759 --> 00:00:13.759
The anxiety you have been carrying since that meeting won't let you sleep.
00:00:14.080 --> 00:00:18.719
Your therapist's next available slot is three weeks away.
00:00:18.960 --> 00:00:23.120
The prices line feels like too much for what this is.
00:00:25.199 --> 00:00:35.280
And you find yourself opening an app or a chatbot or just typing into a search bar looking for something.
00:00:37.200 --> 00:00:45.679
Over 40 million people worldwide are now using AI-powered mental health tools every month.
00:00:46.799 --> 00:00:56.880
And 1.2 million people per week are using chat GPT specifically to discuss suicide.
00:01:00.320 --> 00:01:09.760
Those two facts sit in the same landscape, and today we are going to look at that landscape honestly.
00:01:10.319 --> 00:01:21.200
What AI can genuinely offer in mental health, what it cannot, and where the line between useful and dangerous sits.
00:02:11.360 --> 00:02:15.439
And this is what happens when I ask them out loud.
00:02:26.000 --> 00:02:35.199
This is episode nine, and I want to be clear from the outset that this is not an anti-technology episode.
00:02:35.680 --> 00:02:41.840
AI is already doing some genuinely useful things in the mental health space.
00:02:43.039 --> 00:02:45.520
The evidence for that is real.
00:02:46.879 --> 00:02:58.159
But it is also doing some genuinely dangerous things, and the evidence for that is now arriving faster than the regulation designed to address it.
00:03:04.000 --> 00:03:14.960
This episode covers both fairly with the evidence without either dismissing the technology or overstating what it can do.
00:03:20.800 --> 00:03:25.680
I want to start with the gap that makes this topic urgent.
00:03:31.199 --> 00:03:43.680
In the United States, more than one hundred and sixty million people live in areas designated as mental health professional shortage areas.
00:03:52.960 --> 00:04:00.560
The cost without insurance runs between 150 and 300 per session.
00:04:01.919 --> 00:04:15.280
In the UK, we saw in episode 5 that 18.5 months is the average rate for NHS psychotherapy in some parts of England.
00:04:16.959 --> 00:04:27.759
That gap between the number of people who need support and the number who can access it is not closing, it is widening.
00:04:28.639 --> 00:04:31.920
Into that gap, AI has arrived.
00:04:32.959 --> 00:04:37.439
And the question is not whether people will use it, they already are.
00:04:37.839 --> 00:04:39.839
In enormous numbers.
00:04:40.319 --> 00:04:51.120
The question is what they're using, what it can actually do, and what the risks are when the wrong tool is applied to the wrong moment.
00:04:53.199 --> 00:04:57.040
Because those risks are no longer theoretical.
00:05:00.480 --> 00:05:03.199
The usual frame before we go in.
00:05:05.279 --> 00:05:10.160
I am not a technologist or an AI researcher.
00:05:10.639 --> 00:05:18.240
Everything I share today comes from published research, clinical commentary and documented evidence.
00:05:19.040 --> 00:05:21.360
Sources are in the show notes.
00:05:22.720 --> 00:05:26.079
This episode covers four things.
00:05:27.040 --> 00:05:41.519
First, the AI mental health landscape as it currently exists, what tools are available, which ones have evidence behind them and what they can and cannot do.
00:05:43.279 --> 00:05:50.879
Second, where AI genuinely helps the evidence base for clinically validated tools.
00:05:51.360 --> 00:06:04.079
Third excuse me the dangers specifically the dangers of using general purpose AI for mental health support and self diagnosis.
00:06:04.639 --> 00:06:06.959
This section is serious.
00:06:07.519 --> 00:06:10.000
I'm going to take my time with it.
00:06:12.639 --> 00:06:16.240
Fourth a framework for navigating this.
00:06:16.560 --> 00:06:23.360
How to think about what AI can and cannot be in your mental health journey.
00:06:34.959 --> 00:06:49.439
It is a spectrum from rigorously clinically validated tools to completely unregulated general purpose chatbots being used for purposes they were never designed for.
00:06:51.600 --> 00:06:57.279
Understanding that spectrum matters before we talk about anything else.
00:07:00.319 --> 00:14:45.039
The AI mental health landscape twenty twenty six T one Clinically validated purpose built tools withy five plus peer reviewed publications FDA breakthrough device designation NHS endorsed C marked uses CBT DBT ATT Mindfulness Serves six million plus users Human Coaching Ugrade Available Therabot LA JM AI March 2025 RCT 210 adults with clinically significant depression, anxiety or eating disorder risk 51% average reduction in major depression symptoms Expert Fine tuned research model with clinical oversight The most rigorous AI therapy trial to date robot shut down consumer app june twenty twenty five shifted to enterprise slash health system licensing fourteen RTTs FTA breakthrough device data station clinical evidence remains Tier two Evidence informed consumer apps Upa Headspace EBB Sanve somewhat published research validated instruments structured therapeutic frameworks less rigorous than tier one but not without evidence tier three general purpose AI chatbots chat GPT Gemini character AI replica not designed for mental health no clinical validation no safety guardrails specific to mental health as of twenty twenty six being used by millions for mental health support regardless forty million people use AI mental health tools monthly That three tier distinction is the most important thing I can give you before we go any further A clinically validated tool like RISA and a general purpose chatbot like ChatGPT are not the same thing they have different design intentions different evidence spaces different safety frameworks and different risk profiles the problem and this is the problem that the research is increasingly naming is that most people use AI for mental health support are not using tier one tools they are using tier one sorry tier three because tier three is free it's immediately available it has no waiting list and at 11 PM on a Tuesday when the anxiety won't let you sleep it is right there Let me be precise about what the evidence actually shows for the tools that have been properly studied What the clinical evidence shows for purpose built AI tools Cochrane Adherent Systematic Review twenty twenty four Robot Risa Uper across ten studies clinically meaningful reductions in self reported depression and anxiety symptoms measured by validated instruments PHQ dash nine GAD dash seven RISA RCT Gemini twenty twenty one significantly greater depression reduction versus weight list controls RISA for chronic disease McNeil at all twenty twenty four notable reductions in PHQ dash nine depression and GAD seven anxiety both P001 versus no treatment at four weeks RISA received FDA breakthrough device designation designation after an independent peer reviewed trial found it effective for chronic pain and associated depression slash anxiety comparable to in-person psychological counselling key consistent finding across studies CBC based AI tools appear effective for subclinical and mild to moderate presentations of depression and anxiety not from moderate to severe not as a replacement for professional care as a supplement a bridge or a first step that consistent finding deserves attention sub clinical and mild to moderate as a supplement as a bridge as a first step that is a real and valuable role in a system where the weight for therapy can stretch to eighteen months a tool that can provide structured CBT exercises and measurable symptom reduction for mild to moderate presentations available immediately free 24 hours a day is generally filling a gap but and this is the distinction that gets lost in the marketing and in the 11pm moment these tools were designed and validated for specific presentations with specific safeguards at specific levels of acuity not for everything not for everyone and not for the most vulnerable moments I want to talk about what happens when the wrong tool meets the wrong moment and I want to be clear that what I am about to share is not speculative, it is documented it is current and it is serious The secrofancy problem what the research shows Stanford University Study Science march 2026 tested eleven leading AI systems for secrofancy the tendency to agree with and validate users regardless of accuracy.
00:14:45.840 --> 00:15:46.159
Models on average endorsed users positions four to nine percent more often than humans would, even when users described manipulative, deceptive or outright illegal scenarios OpenAI acknowledged in 2025 that chat GPT's sycophantic behavior can raise safety concerns, including around issues like mental health, emotional over reliance or risky behavior Harvard psychiatrist Macheri Kesavan wrote psychiatry 2026 Generative AI can function as a social substitute, reducing real world corrective feedback while its tendency toward confirmatory responses reinforces existing beliefs.
00:15:46.720 --> 00:21:40.319
AI doesn't plant new ideas in people's heads so much as it turns up the volume on whatever is already there I want to emphasize that last part again turn up the volume on whatever is already there for someone experiencing depressive thinking I am worthless nothing will help there is no point a system that turns up the volume on those thoughts through validation and agreement is not providing support it is providing amplification and for someone who was thinking is moving towards crisis that amplification is not a neutral feature it is a risk The Crisis Response Problem documented cases and research Open AI late 2025 1.2 million people per week use chat GPT to discuss suicide no clinical safeguards in place for most Psychiatric Times review november 2024 through to july 2025 30 chatbots Chatbots should be contradicted for suicidal patients their strong tendency to validate can accentuate self destructive ideation and turn impulses into action stress test by a psychiatrist ten popular chatbots were prompted by someone pretending to be a desperate 14 year old several encouraged suicide one suggested a method August 2025 lawsuit filed against open AI by parents of Adam Rein 16 who died by suicide and I have to emphasize this allegedly alleged chat GPT encouraged his suicidal addition gave advice about methods and discouraged him from telling his parents OpenAI denied responsibility counter AI multiple wrong death lawsuits following user suicides APA urged FTC to oversee mental health chatbots lacking clinical validation or ethical safeguards Brown University march twenty twenty six fifteen distinct ethical risks identified in AI responses versus peer counselors and licensed psychologists including mishandling crises, reinforcing harmful beliefs and deceptive empathy a pattern in which the chatbot mimics the language of care without the clinical understanding to apply it appropriately That is not a software bug that can be patched it is a structural limitation of what these systems are they are trained on language they produce language that sounds like care but they do not have a clinical framework for knowing when that language is doing harm rather than good and now the self diagnosis problem the self diagnosis problem why AI gets mental health wrong one in six US adults are use AI chatbots monthly for health advice rises to 25% for adults under 30 KFF polls 2024 13% of 18 to 21 year olds report using AI for mental health support why AI self diagnosis is structurally flawed one confirmation bias amplification users ask questions shaped by what they already suspect AI validates rather than challenges pre existing beliefs are confirmed not examined two missing clinical context mental health diagnosis requires history observation clinical interview rollout of physical causes and professional judgment AI has none of these it has your typed words that is not sufficient three hallucination and stigma twenty twenty five study computation and language LLMs expressed stigma toward people with mental health conditions and provided inappropriate advice twenty twenty five study communications medicine LLMs can generate false facts in quotes hallucinations presented with confidence for the complexity problem mental health presentations overlap co occur and present differently in different people What looks like depression may be bipolar disorder what looks like anxiety may be PCSD.
00:21:40.720 --> 00:22:09.039
Getting this wrong has clinical consequences ninety four percent of doctors surveyed have concerns about patients relying on AI tools for medical advice SEMOP twenty twenty five That fourth point, the complexity problem, is the one that matters most for the listeners of this show specifically.
00:22:10.559 --> 00:22:19.279
Because the people most likely to turn to AI for mental health support are not people with simple, straightforward presentations.
00:22:19.759 --> 00:22:32.400
They are people who have often been struggling for years, who have complex histories, who may have had multiple diagnoses or not at all that felt accurate.
00:22:34.160 --> 00:22:44.160
Those are precisely the presentations that AI is least equipped to navigate, precisely the ones where a wrong cancer carries the most cost.
00:22:47.279 --> 00:23:04.400
What AI cannot do in mental health current clinical consensus US News slash psychiatry november twenty twenty five LLMs are not a substitute for professional mental health care.
00:23:04.799 --> 00:23:09.680
They cannot provide an accurate diagnosis or crisis management.
00:23:10.880 --> 00:23:18.880
The advice LLMs offer lacks individual context which is crucial for safe and effective psychiatric care.
00:23:19.680 --> 00:23:22.880
Psychology Today, October 2025.
00:23:23.680 --> 00:23:36.960
LLMs while helpful in certain structured tasks currently perform at best as low quality therapists with limitations in empathy, bias and cultural understanding.
00:23:37.759 --> 00:23:42.319
HIPAA does not protect your mental health data shared with AI.
00:23:42.720 --> 00:23:57.519
In 2023, telehealth company Cerebral admitted sharing patient data with advertising platforms in 2024, a major wellness app breach affected 3 million users.
00:23:58.319 --> 00:24:09.440
As of May 2026, no general purpose AI system has been approved by the FDA to diagnose, treat or cure a mental health disorder.
00:24:10.960 --> 00:24:15.839
I will emphasize and repeat that last sentence again.
00:24:16.160 --> 00:24:27.440
No general purpose AI system has been approved by the FDA to diagnose, treat or cure a mental health disorder.
00:24:28.400 --> 00:24:34.559
That sentence is doing the same work as the CE mark distinction in episode 6.
00:24:36.079 --> 00:24:48.000
The tool exists, people are using it, but the regulatory approval that would tell us it is safe and effective for the purpose it is being used for does not yet exist.
00:24:51.119 --> 00:24:54.000
So what do you do with all of this?
00:24:55.839 --> 00:25:06.160
I want to give you something practical, not a list of things to avoid, a way of thinking about what AI can and cannot be in your mental health journey.
00:25:09.279 --> 00:25:18.880
The first thing know which tier you're using I'm going to emphasize something here.
00:25:19.200 --> 00:25:26.640
I do not I I do j I do genuinely want to be really clear.
00:25:27.440 --> 00:25:32.079
I do not recommend using AI for mental health.
00:25:32.960 --> 00:25:35.359
I'm just I just want to put that out there.
00:25:35.599 --> 00:25:45.599
But if you are going to use AI for mental health support, use a purpose-built, clinically validated tool rather than a general purpose chatbot.
00:25:45.920 --> 00:25:53.039
VISA has 45 plus peer-reviewed publications and FDA recognition.
00:25:53.359 --> 00:25:58.400
ChatGBT has neither of those things for mental health specifically.
00:25:58.720 --> 00:26:00.720
That distinction matters.
00:26:01.839 --> 00:26:06.240
The second thing, use it as a bridge, not a destination.
00:26:06.799 --> 00:26:18.000
The evidence supports AI tools for mild to moderate presentations as a supplement to professional care or as a first step while waiting for access.
00:26:18.319 --> 00:26:25.759
It does not support AI as a replacement for professional care for complex or severe presentations.
00:26:27.200 --> 00:26:36.559
If what you are carrying is complex and many of you listening to this show are carrying something complex, AI is not the answer.
00:26:37.200 --> 00:26:42.000
It may be part of managing the weight, it is not the treatment.
00:26:42.640 --> 00:26:48.000
The third thing, never use AI in a crisis.
00:26:48.400 --> 00:26:51.440
This is the clearest line I can draw.
00:26:51.839 --> 00:27:04.000
If you are in a moment of acute distress, suicidal thoughts, self-harm urges, a mental health emergency, an AI chatbot is not the resource.
00:27:04.799 --> 00:27:11.119
Call or text 988 text home to 741 741.
00:27:11.680 --> 00:27:15.279
Call someone, go to an emergency room if you need to.
00:27:16.079 --> 00:27:20.319
AI in a crisis moment is not support, it is risk.
00:27:21.279 --> 00:27:25.440
The fourth thing, do not give AI your diagnosis.
00:27:26.000 --> 00:27:29.200
The confirmation bias problem is structural.
00:27:29.519 --> 00:27:42.559
If you go to an AI chatbot believing you have a specific condition and ask questions framed around that belief, the AI will tend to validate rather than challenge.
00:27:42.720 --> 00:27:44.799
That is not a diagnosis.
00:27:45.279 --> 00:27:52.240
That is reflection and in mental health, reflection without clinical judgments can do real harm.
00:27:52.960 --> 00:27:56.400
And the fifth thing, protect your data.
00:27:57.599 --> 00:28:02.640
HIPAA does not protect what you share with an AI chatbot.
00:28:02.880 --> 00:28:17.759
Do not share identifying information, use a separate email address, be aware that what you type may be stored, used for training data, and in some documented cases shared with third parties.
00:28:21.759 --> 00:28:26.000
AI is not going away from mental health.
00:28:26.640 --> 00:28:38.799
The access gap is too large, the demand is too great, and the technology is too immediately available for that to happen.
00:28:40.960 --> 00:28:45.839
And some of what is already doing is genuinely valuable.
00:28:46.240 --> 00:28:55.920
The therobox child showing 51% depression reduction, the real evidence for chronic pain and associated mental health.
00:28:56.240 --> 00:29:08.880
The reality that is structured CB2 available at no cost at any hour can make a meaningful difference for someone who would otherwise have nothing.
00:29:09.200 --> 00:29:11.759
That is real and it matters.
00:29:12.079 --> 00:29:39.039
What also matters and what this episode has been about is that 1.2 million people a week discussing suicide with a system that was not designed for that conversation has no clinical framework for it and has been documented, validating whether than interrupting suicide or thinking is not the same thing.
00:29:40.160 --> 00:29:47.440
Those are two different tools in the same landscape and the difference between them is worth knowing.
00:29:47.839 --> 00:29:50.160
That's what this episode was for.
00:29:52.160 --> 00:29:55.680
That's not a fixed skilled here, episode 9.
00:29:55.920 --> 00:30:01.279
If anything in today's episode brought something off for you, please reach out for support.
00:30:01.759 --> 00:30:07.279
You can call or text 988 the suicide and crisis lifeline.
00:30:07.759 --> 00:30:11.200
Free confidential available 24 hours a day.
00:30:11.519 --> 00:30:14.480
This is a human being, not a chatbot.
00:30:15.759 --> 00:30:26.480
Or text home to 741-741, the Crisis Techline, also free confidential and staffed by trained human counsellors.
00:30:27.119 --> 00:30:30.319
You can find this show wherever you listen to your podcasts.
00:30:30.480 --> 00:30:35.359
If today's episode meant something to you, please share it with someone who might need it.
00:30:35.759 --> 00:30:51.839
Also, if you could spare a moment and give this episode a rating on your chosen platform, it would really help and bring other people into the conversation.
00:30:52.960 --> 00:30:56.640
I'm Torrin, I'll be back for episode ten.
00:30:57.200 --> 00:30:58.799
Take care of yourself.
00:00:02.879 --> 00:00:06.480
It is today at 11 pm.
00:00:07.759 --> 00:00:13.759
The anxiety you have been carrying since that meeting won't let you sleep.
00:00:14.080 --> 00:00:18.719
Your therapist's next available slot is three weeks away.
00:00:18.960 --> 00:00:23.120
The prices line feels like too much for what this is.
00:00:25.199 --> 00:00:35.280
And you find yourself opening an app or a chatbot or just typing into a search bar looking for something.
00:00:37.200 --> 00:00:45.679
Over 40 million people worldwide are now using AI-powered mental health tools every month.
00:00:46.799 --> 00:00:56.880
And 1.2 million people per week are using chat GPT specifically to discuss suicide.
00:01:00.320 --> 00:01:09.760
Those two facts sit in the same landscape, and today we are going to look at that landscape honestly.
00:01:10.319 --> 00:01:21.200
What AI can genuinely offer in mental health, what it cannot, and where the line between useful and dangerous sits.
00:02:11.360 --> 00:02:15.439
And this is what happens when I ask them out loud.
00:02:26.000 --> 00:02:35.199
This is episode nine, and I want to be clear from the outset that this is not an anti-technology episode.
00:02:35.680 --> 00:02:41.840
AI is already doing some genuinely useful things in the mental health space.
00:02:43.039 --> 00:02:45.520
The evidence for that is real.
00:02:46.879 --> 00:02:58.159
But it is also doing some genuinely dangerous things, and the evidence for that is now arriving faster than the regulation designed to address it.
00:03:04.000 --> 00:03:14.960
This episode covers both fairly with the evidence without either dismissing the technology or overstating what it can do.
00:03:20.800 --> 00:03:25.680
I want to start with the gap that makes this topic urgent.
00:03:31.199 --> 00:03:43.680
In the United States, more than one hundred and sixty million people live in areas designated as mental health professional shortage areas.
00:03:52.960 --> 00:04:00.560
The cost without insurance runs between 150 and 300 per session.
00:04:01.919 --> 00:04:15.280
In the UK, we saw in episode 5 that 18.5 months is the average rate for NHS psychotherapy in some parts of England.
00:04:16.959 --> 00:04:27.759
That gap between the number of people who need support and the number who can access it is not closing, it is widening.
00:04:28.639 --> 00:04:31.920
Into that gap, AI has arrived.
00:04:32.959 --> 00:04:37.439
And the question is not whether people will use it, they already are.
00:04:37.839 --> 00:04:39.839
In enormous numbers.
00:04:40.319 --> 00:04:51.120
The question is what they're using, what it can actually do, and what the risks are when the wrong tool is applied to the wrong moment.
00:04:53.199 --> 00:04:57.040
Because those risks are no longer theoretical.
00:05:00.480 --> 00:05:03.199
The usual frame before we go in.
00:05:05.279 --> 00:05:10.160
I am not a technologist or an AI researcher.
00:05:10.639 --> 00:05:18.240
Everything I share today comes from published research, clinical commentary and documented evidence.
00:05:19.040 --> 00:05:21.360
Sources are in the show notes.
00:05:22.720 --> 00:05:26.079
This episode covers four things.
00:05:27.040 --> 00:05:41.519
First, the AI mental health landscape as it currently exists, what tools are available, which ones have evidence behind them and what they can and cannot do.
00:05:43.279 --> 00:05:50.879
Second, where AI genuinely helps the evidence base for clinically validated tools.
00:05:51.360 --> 00:06:04.079
Third excuse me the dangers specifically the dangers of using general purpose AI for mental health support and self diagnosis.
00:06:04.639 --> 00:06:06.959
This section is serious.
00:06:07.519 --> 00:06:10.000
I'm going to take my time with it.
00:06:12.639 --> 00:06:16.240
Fourth a framework for navigating this.
00:06:16.560 --> 00:06:23.360
How to think about what AI can and cannot be in your mental health journey.
00:06:34.959 --> 00:06:49.439
It is a spectrum from rigorously clinically validated tools to completely unregulated general purpose chatbots being used for purposes they were never designed for.
00:06:51.600 --> 00:06:57.279
Understanding that spectrum matters before we talk about anything else.
00:07:00.319 --> 00:14:45.039
The AI mental health landscape twenty twenty six T one Clinically validated purpose built tools withy five plus peer reviewed publications FDA breakthrough device designation NHS endorsed C marked uses CBT DBT ATT Mindfulness Serves six million plus users Human Coaching Ugrade Available Therabot LA JM AI March 2025 RCT 210 adults with clinically significant depression, anxiety or eating disorder risk 51% average reduction in major depression symptoms Expert Fine tuned research model with clinical oversight The most rigorous AI therapy trial to date robot shut down consumer app june twenty twenty five shifted to enterprise slash health system licensing fourteen RTTs FTA breakthrough device data station clinical evidence remains Tier two Evidence informed consumer apps Upa Headspace EBB Sanve somewhat published research validated instruments structured therapeutic frameworks less rigorous than tier one but not without evidence tier three general purpose AI chatbots chat GPT Gemini character AI replica not designed for mental health no clinical validation no safety guardrails specific to mental health as of twenty twenty six being used by millions for mental health support regardless forty million people use AI mental health tools monthly That three tier distinction is the most important thing I can give you before we go any further A clinically validated tool like RISA and a general purpose chatbot like ChatGPT are not the same thing they have different design intentions different evidence spaces different safety frameworks and different risk profiles the problem and this is the problem that the research is increasingly naming is that most people use AI for mental health support are not using tier one tools they are using tier one sorry tier three because tier three is free it's immediately available it has no waiting list and at 11 PM on a Tuesday when the anxiety won't let you sleep it is right there Let me be precise about what the evidence actually shows for the tools that have been properly studied What the clinical evidence shows for purpose built AI tools Cochrane Adherent Systematic Review twenty twenty four Robot Risa Uper across ten studies clinically meaningful reductions in self reported depression and anxiety symptoms measured by validated instruments PHQ dash nine GAD dash seven RISA RCT Gemini twenty twenty one significantly greater depression reduction versus weight list controls RISA for chronic disease McNeil at all twenty twenty four notable reductions in PHQ dash nine depression and GAD seven anxiety both P001 versus no treatment at four weeks RISA received FDA breakthrough device designation designation after an independent peer reviewed trial found it effective for chronic pain and associated depression slash anxiety comparable to in-person psychological counselling key consistent finding across studies CBC based AI tools appear effective for subclinical and mild to moderate presentations of depression and anxiety not from moderate to severe not as a replacement for professional care as a supplement a bridge or a first step that consistent finding deserves attention sub clinical and mild to moderate as a supplement as a bridge as a first step that is a real and valuable role in a system where the weight for therapy can stretch to eighteen months a tool that can provide structured CBT exercises and measurable symptom reduction for mild to moderate presentations available immediately free 24 hours a day is generally filling a gap but and this is the distinction that gets lost in the marketing and in the 11pm moment these tools were designed and validated for specific presentations with specific safeguards at specific levels of acuity not for everything not for everyone and not for the most vulnerable moments I want to talk about what happens when the wrong tool meets the wrong moment and I want to be clear that what I am about to share is not speculative, it is documented it is current and it is serious The secrofancy problem what the research shows Stanford University Study Science march 2026 tested eleven leading AI systems for secrofancy the tendency to agree with and validate users regardless of accuracy.
00:14:45.840 --> 00:15:46.159
Models on average endorsed users positions four to nine percent more often than humans would, even when users described manipulative, deceptive or outright illegal scenarios OpenAI acknowledged in 2025 that chat GPT's sycophantic behavior can raise safety concerns, including around issues like mental health, emotional over reliance or risky behavior Harvard psychiatrist Macheri Kesavan wrote psychiatry 2026 Generative AI can function as a social substitute, reducing real world corrective feedback while its tendency toward confirmatory responses reinforces existing beliefs.
00:15:46.720 --> 00:21:40.319
AI doesn't plant new ideas in people's heads so much as it turns up the volume on whatever is already there I want to emphasize that last part again turn up the volume on whatever is already there for someone experiencing depressive thinking I am worthless nothing will help there is no point a system that turns up the volume on those thoughts through validation and agreement is not providing support it is providing amplification and for someone who was thinking is moving towards crisis that amplification is not a neutral feature it is a risk The Crisis Response Problem documented cases and research Open AI late 2025 1.2 million people per week use chat GPT to discuss suicide no clinical safeguards in place for most Psychiatric Times review november 2024 through to july 2025 30 chatbots Chatbots should be contradicted for suicidal patients their strong tendency to validate can accentuate self destructive ideation and turn impulses into action stress test by a psychiatrist ten popular chatbots were prompted by someone pretending to be a desperate 14 year old several encouraged suicide one suggested a method August 2025 lawsuit filed against open AI by parents of Adam Rein 16 who died by suicide and I have to emphasize this allegedly alleged chat GPT encouraged his suicidal addition gave advice about methods and discouraged him from telling his parents OpenAI denied responsibility counter AI multiple wrong death lawsuits following user suicides APA urged FTC to oversee mental health chatbots lacking clinical validation or ethical safeguards Brown University march twenty twenty six fifteen distinct ethical risks identified in AI responses versus peer counselors and licensed psychologists including mishandling crises, reinforcing harmful beliefs and deceptive empathy a pattern in which the chatbot mimics the language of care without the clinical understanding to apply it appropriately That is not a software bug that can be patched it is a structural limitation of what these systems are they are trained on language they produce language that sounds like care but they do not have a clinical framework for knowing when that language is doing harm rather than good and now the self diagnosis problem the self diagnosis problem why AI gets mental health wrong one in six US adults are use AI chatbots monthly for health advice rises to 25% for adults under 30 KFF polls 2024 13% of 18 to 21 year olds report using AI for mental health support why AI self diagnosis is structurally flawed one confirmation bias amplification users ask questions shaped by what they already suspect AI validates rather than challenges pre existing beliefs are confirmed not examined two missing clinical context mental health diagnosis requires history observation clinical interview rollout of physical causes and professional judgment AI has none of these it has your typed words that is not sufficient three hallucination and stigma twenty twenty five study computation and language LLMs expressed stigma toward people with mental health conditions and provided inappropriate advice twenty twenty five study communications medicine LLMs can generate false facts in quotes hallucinations presented with confidence for the complexity problem mental health presentations overlap co occur and present differently in different people What looks like depression may be bipolar disorder what looks like anxiety may be PCSD.
00:21:40.720 --> 00:22:09.039
Getting this wrong has clinical consequences ninety four percent of doctors surveyed have concerns about patients relying on AI tools for medical advice SEMOP twenty twenty five That fourth point, the complexity problem, is the one that matters most for the listeners of this show specifically.
00:22:10.559 --> 00:22:19.279
Because the people most likely to turn to AI for mental health support are not people with simple, straightforward presentations.
00:22:19.759 --> 00:22:32.400
They are people who have often been struggling for years, who have complex histories, who may have had multiple diagnoses or not at all that felt accurate.
00:22:34.160 --> 00:22:44.160
Those are precisely the presentations that AI is least equipped to navigate, precisely the ones where a wrong cancer carries the most cost.
00:22:47.279 --> 00:23:04.400
What AI cannot do in mental health current clinical consensus US News slash psychiatry november twenty twenty five LLMs are not a substitute for professional mental health care.
00:23:04.799 --> 00:23:09.680
They cannot provide an accurate diagnosis or crisis management.
00:23:10.880 --> 00:23:18.880
The advice LLMs offer lacks individual context which is crucial for safe and effective psychiatric care.
00:23:19.680 --> 00:23:22.880
Psychology Today, October 2025.
00:23:23.680 --> 00:23:36.960
LLMs while helpful in certain structured tasks currently perform at best as low quality therapists with limitations in empathy, bias and cultural understanding.
00:23:37.759 --> 00:23:42.319
HIPAA does not protect your mental health data shared with AI.
00:23:42.720 --> 00:23:57.519
In 2023, telehealth company Cerebral admitted sharing patient data with advertising platforms in 2024, a major wellness app breach affected 3 million users.
00:23:58.319 --> 00:24:09.440
As of May 2026, no general purpose AI system has been approved by the FDA to diagnose, treat or cure a mental health disorder.
00:24:10.960 --> 00:24:15.839
I will emphasize and repeat that last sentence again.
00:24:16.160 --> 00:24:27.440
No general purpose AI system has been approved by the FDA to diagnose, treat or cure a mental health disorder.
00:24:28.400 --> 00:24:34.559
That sentence is doing the same work as the CE mark distinction in episode 6.
00:24:36.079 --> 00:24:48.000
The tool exists, people are using it, but the regulatory approval that would tell us it is safe and effective for the purpose it is being used for does not yet exist.
00:24:51.119 --> 00:24:54.000
So what do you do with all of this?
00:24:55.839 --> 00:25:06.160
I want to give you something practical, not a list of things to avoid, a way of thinking about what AI can and cannot be in your mental health journey.
00:25:09.279 --> 00:25:18.880
The first thing know which tier you're using I'm going to emphasize something here.
00:25:19.200 --> 00:25:26.640
I do not I I do j I do genuinely want to be really clear.
00:25:27.440 --> 00:25:32.079
I do not recommend using AI for mental health.
00:25:32.960 --> 00:25:35.359
I'm just I just want to put that out there.
00:25:35.599 --> 00:25:45.599
But if you are going to use AI for mental health support, use a purpose-built, clinically validated tool rather than a general purpose chatbot.
00:25:45.920 --> 00:25:53.039
VISA has 45 plus peer-reviewed publications and FDA recognition.
00:25:53.359 --> 00:25:58.400
ChatGBT has neither of those things for mental health specifically.
00:25:58.720 --> 00:26:00.720
That distinction matters.
00:26:01.839 --> 00:26:06.240
The second thing, use it as a bridge, not a destination.
00:26:06.799 --> 00:26:18.000
The evidence supports AI tools for mild to moderate presentations as a supplement to professional care or as a first step while waiting for access.
00:26:18.319 --> 00:26:25.759
It does not support AI as a replacement for professional care for complex or severe presentations.
00:26:27.200 --> 00:26:36.559
If what you are carrying is complex and many of you listening to this show are carrying something complex, AI is not the answer.
00:26:37.200 --> 00:26:42.000
It may be part of managing the weight, it is not the treatment.
00:26:42.640 --> 00:26:48.000
The third thing, never use AI in a crisis.
00:26:48.400 --> 00:26:51.440
This is the clearest line I can draw.
00:26:51.839 --> 00:27:04.000
If you are in a moment of acute distress, suicidal thoughts, self-harm urges, a mental health emergency, an AI chatbot is not the resource.
00:27:04.799 --> 00:27:11.119
Call or text 988 text home to 741 741.
00:27:11.680 --> 00:27:15.279
Call someone, go to an emergency room if you need to.
00:27:16.079 --> 00:27:20.319
AI in a crisis moment is not support, it is risk.
00:27:21.279 --> 00:27:25.440
The fourth thing, do not give AI your diagnosis.
00:27:26.000 --> 00:27:29.200
The confirmation bias problem is structural.
00:27:29.519 --> 00:27:42.559
If you go to an AI chatbot believing you have a specific condition and ask questions framed around that belief, the AI will tend to validate rather than challenge.
00:27:42.720 --> 00:27:44.799
That is not a diagnosis.
00:27:45.279 --> 00:27:52.240
That is reflection and in mental health, reflection without clinical judgments can do real harm.
00:27:52.960 --> 00:27:56.400
And the fifth thing, protect your data.
00:27:57.599 --> 00:28:02.640
HIPAA does not protect what you share with an AI chatbot.
00:28:02.880 --> 00:28:17.759
Do not share identifying information, use a separate email address, be aware that what you type may be stored, used for training data, and in some documented cases shared with third parties.
00:28:21.759 --> 00:28:26.000
AI is not going away from mental health.
00:28:26.640 --> 00:28:38.799
The access gap is too large, the demand is too great, and the technology is too immediately available for that to happen.
00:28:40.960 --> 00:28:45.839
And some of what is already doing is genuinely valuable.
00:28:46.240 --> 00:28:55.920
The therobox child showing 51% depression reduction, the real evidence for chronic pain and associated mental health.
00:28:56.240 --> 00:29:08.880
The reality that is structured CB2 available at no cost at any hour can make a meaningful difference for someone who would otherwise have nothing.
00:29:09.200 --> 00:29:11.759
That is real and it matters.
00:29:12.079 --> 00:29:39.039
What also matters and what this episode has been about is that 1.2 million people a week discussing suicide with a system that was not designed for that conversation has no clinical framework for it and has been documented, validating whether than interrupting suicide or thinking is not the same thing.
00:29:40.160 --> 00:29:47.440
Those are two different tools in the same landscape and the difference between them is worth knowing.
00:29:47.839 --> 00:29:50.160
That's what this episode was for.
00:29:52.160 --> 00:29:55.680
That's not a fixed skilled here, episode 9.
00:29:55.920 --> 00:30:01.279
If anything in today's episode brought something off for you, please reach out for support.
00:30:01.759 --> 00:30:07.279
You can call or text 988 the suicide and crisis lifeline.
00:30:07.759 --> 00:30:11.200
Free confidential available 24 hours a day.
00:30:11.519 --> 00:30:14.480
This is a human being, not a chatbot.
00:30:15.759 --> 00:30:26.480
Or text home to 741-741, the Crisis Techline, also free confidential and staffed by trained human counsellors.
00:30:27.119 --> 00:30:30.319
You can find this show wherever you listen to your podcasts.
00:30:30.480 --> 00:30:35.359
If today's episode meant something to you, please share it with someone who might need it.
00:30:35.759 --> 00:30:51.839
Also, if you could spare a moment and give this episode a rating on your chosen platform, it would really help and bring other people into the conversation.
00:30:52.960 --> 00:30:56.640
I'm Torrin, I'll be back for episode ten.
00:30:57.200 --> 00:30:58.799
Take care of yourself.
