WEBVTT
00:01:01.340 --> 00:01:04.299
The following is a site to practice production.
00:01:14.540 --> 00:01:18.620
Welcome to Beyond the Bot AI and Mental Health.
00:01:18.780 --> 00:01:28.620
I'm Dan Florel, a professor at Eastern Kentucky University, and we are going to be continuing on this episode to talk about AI report writing.
00:01:28.859 --> 00:01:48.859
If you recall from our last episode, we got into a little bit of the weeds, really into AI report recommendations and a little bit into how to format different AI reports that are from the same student or for the same client for different audiences and kind of focused on that.
00:01:49.020 --> 00:02:00.379
So this time around, we're going to be more focusing on kind of the levels of use that you could still argue are AI-assisted reports and what those exactly look like.
00:02:00.539 --> 00:02:05.740
But before we get into that, I want to introduce my co-host and partner in crime here.
00:02:06.379 --> 00:02:13.419
Adam, why don't you go ahead and introduce yourself and then we can get launched into more AI report writing discussion.
00:02:14.379 --> 00:02:15.259
Thanks, Dan.
00:02:15.419 --> 00:02:18.379
Adam Lockwood at Kent State University.
00:02:19.020 --> 00:02:21.740
Conflict of interest, same as before.
00:02:22.060 --> 00:02:26.300
Report writing, subject matter expert for uh for par.
00:02:28.140 --> 00:02:28.780
Yep.
00:02:30.140 --> 00:02:36.300
So we're gonna rely on people listening to uh our episodes sequentially as they build up.
00:02:36.620 --> 00:02:47.900
But you know, I think when we were talking about AI report writing, this is a huge issue, one that people continually talk to us about in our presentations.
00:02:48.140 --> 00:02:51.900
And fortunately, we are getting more knowledge about what those look like.
00:02:51.979 --> 00:02:55.900
And of course, there's a lot more products out there than ever before.
00:02:56.060 --> 00:03:17.419
And I just kind of go by if you've gone to the NAS convention and you go down to the vending area over the last three years, it's like they are exponentially more and more new report writers, and frankly, ones that I think are better every year, at least what they claim to be able to do, uh, versus what the first ones three years ago were claiming.
00:03:17.579 --> 00:03:18.859
So real impressive.
00:03:19.019 --> 00:03:28.780
But I think we need to get a little into the weeds, a little more general versus particular products, and that is exactly what are we considering AI assisted report writing?
00:03:28.939 --> 00:03:44.139
Adam, why don't you, because you run a lot of the research that we do in the field on AI, what would you consider kind of these different levels that we look at in regards to an AI assist for our reports?
00:03:44.460 --> 00:03:50.299
Well, I mean, honestly, we've been using AI and our report writing for quite some time.
00:03:50.859 --> 00:03:55.259
So at the most basic level, it's things like Grammarly.
00:03:55.500 --> 00:04:05.739
You know, technically, Grammarly is artificial intelligence and it's natural language processing, which is a similar technology to Chat GPT.
00:04:07.099 --> 00:04:19.980
One thing I always want to point out to people, and that a few people are caught off guard by, is that you should not be using Grammarly to help you with your reports unless you have an enterprise account.
00:04:20.860 --> 00:04:24.139
Because it trains on the data you provide it.
00:04:24.540 --> 00:04:34.780
So if you're using it just for grammar, spelling, and editing, it's taking your sensitive data and using it for who knows what.
00:04:35.019 --> 00:04:56.139
So that's the first level is this kind of grammarly, uh grammar spelling checking typos, and um that's pretty low stakes from a um clinical judgment, loss of skills uh perspective, but it can be high stakes from a data security, FERPA, HIPAA standpoint.
00:04:56.379 --> 00:05:01.980
So um that's that's kind of like the first level I think of when I think of AI.
00:05:02.860 --> 00:05:08.300
Yeah, so grammarly I get that, but what about like Microsoft Word, Google Docs?
00:05:08.540 --> 00:05:14.379
Because they do offer grammar, you know, punctuation spelling, you know, help.
00:05:14.860 --> 00:05:17.820
That's not really is that still considered AI?
00:05:17.900 --> 00:05:23.260
And would the same concerns apply when you're using those to write up the reports?
00:05:23.580 --> 00:05:27.340
That's gonna depend on a case-by-case basis.
00:05:27.580 --> 00:05:36.140
You know, spell check typically, my understanding, I'm not a computer programmer, but it's not using this process.
00:05:36.700 --> 00:05:47.340
It's also not underlying sentences and being like, this sentence is a fragment, or you know, consider rewording and giving you suggestions.
00:05:47.820 --> 00:05:50.780
Um so it's a little bit different.
00:05:50.939 --> 00:05:55.260
At this point, though, AI is inserted into everything, and you need to be careful.
00:05:55.420 --> 00:06:13.660
You know, if you use Google, there's a good chance that Gemini, which is generative AI, is integrated into your Google Docs, into your spreadsheets, into every single component.
00:06:13.820 --> 00:06:17.020
The same for Word, Copilot is up in everything.
00:06:17.260 --> 00:06:29.660
So if you're a Google workspace, a Google campus or a um Microsoft campus, you should definitely find out is do we have an enterprise account?
00:06:29.820 --> 00:06:32.939
And if so, what can I use it?
00:06:33.020 --> 00:06:38.060
Um, what's what's ask someone in IT or legal or administration?
00:06:38.300 --> 00:06:44.300
Is it okay for me to be using Gemini Integrated, or is it okay for me to be using Copilot?
00:06:44.540 --> 00:06:52.460
Well, I was gonna say because you know, back in the day, obviously Word and maybe some to lesser extent Google Docs, I mean it resided on your computer.
00:06:52.620 --> 00:06:57.020
You could store it there, and Microsoft kind of lost control of it for the most part afterwards.
00:06:57.100 --> 00:07:05.020
But now they've put everything into Microsoft 360, and of course, Google has always resided into these cloud servers.
00:07:05.260 --> 00:07:14.939
And then, of course, they've been integrating their premier product, whether it's co-pilot with Microsoft or Gemini with uh Google, into the products.
00:07:15.100 --> 00:07:17.740
Like there's really not even a selection process.
00:07:17.900 --> 00:07:33.340
So I guess from a most basic, most conservative view, one should always have the enterprise or business level kind of access that most organizations people work for would have.
00:07:33.500 --> 00:07:46.939
Uh, with the caveat, of course, being if you're a solo private practice, that you probably don't have that level of protection in an individual account, or if you're using the free account, certainly not.
00:07:47.180 --> 00:08:03.340
In which case, I guess you would argue, Adam, that uh maybe they shouldn't be using that or they should look into upgrading to a higher level where their materials aren't being used for training purposes or otherwise captured beyond what you're using it for.
00:08:03.580 --> 00:08:10.540
I've consulted with people in private practice to help them get BAAs in place for Jim and I co-pilot as well.
00:08:10.780 --> 00:08:17.420
It's a pain, but it's definitely doable, and it's something that I would do if I was in private practice for sure.
00:08:18.060 --> 00:08:18.379
Okay.
00:08:18.860 --> 00:08:20.140
And is that something?
00:08:20.300 --> 00:08:20.939
What do you do?
00:08:21.100 --> 00:08:26.780
Ask AI kind of how to go about getting a BAA with those organizations or or services?
00:08:27.180 --> 00:08:32.620
It's been almost a year since the last time I did it, and it always changes on how you do it.
00:08:32.860 --> 00:08:54.059
But um essentially, you know, you get an enterprise account, um, you pay for a certain level of um, you know, a suite that is where they have a business associates agreement with you, they email you a document that says the BAA is in place, and um you know, depending on your level of risk tolerance.
00:08:54.139 --> 00:09:00.059
For me, I think if I had that BAA an email, I would I would be okay, you know, I would be comfortable.
00:09:01.820 --> 00:09:02.059
Right.
00:09:02.460 --> 00:09:10.860
If I had something from Goo, I I always encourage people get an email with the BAA saying that it's HIPAA compliant.
00:09:11.100 --> 00:09:14.700
I'm not a lawyer, so you know, I always give that copy out.
00:09:14.779 --> 00:09:37.180
But I for my own level of comfort, I want something from the company if I'm in private practice, or I want some something from someone in authority in my organization, whether that's IT legal or administration saying, yes, you can use you know this when you're working with students or when you're using this with clients.
00:09:37.980 --> 00:09:39.899
And I'd say that email.
00:09:40.700 --> 00:09:41.420
Yeah, yeah.
00:09:41.580 --> 00:09:44.379
I was gonna say that's one of those things you want documented.
00:09:44.460 --> 00:09:47.899
That's one also the advantage if you're working for a larger organization.
00:09:48.060 --> 00:09:55.180
So, for example, most school districts that have Google that are a Google campus, they'll probably have the enterprise as part of it.
00:09:55.340 --> 00:09:59.580
And what I've heard anyway is that that makes it FERPA compliant.
00:09:59.659 --> 00:10:08.700
Um, I haven't heard about HIPAA compliant, but in the schools, of course, the FERPA law is going to be the one that you would be more attended to anyway.
00:10:09.180 --> 00:10:17.659
I think we've given some people the things to think about in regards to the level of security, regardless of how innocuous the AI seems.
00:10:17.820 --> 00:10:24.940
It's kind of like it's seeped itself into areas that you hadn't previously had to worry about in the same way.
00:10:25.340 --> 00:10:30.540
So at the lowest level, you were talking about grammar, spelling, editing, suggestions.
00:10:30.700 --> 00:10:33.820
It's kind of your low-level AI-assisted report, right?
00:10:34.060 --> 00:10:49.899
Or what kind of levels come next in regards to kind of the gradient of relatively benign, basic writing help to, of course, at the very end would be writing the whole thing, giving a diagnosis and going on kind of thing.
00:10:50.060 --> 00:10:55.340
What are some of the things in between that people might start considering uh adopting?
00:10:55.500 --> 00:10:59.340
And as you know, they're beginning to maybe embrace AI.
00:10:59.580 --> 00:11:16.540
I always tell people, kind of, let's go slowly, uh, you know, gradually integrate it within your practice to a point where you feel like you've either lost some of the specialization and skills that you need yourself, uh, and also where you're comfortable ethically, legally from that standpoint.
00:11:16.860 --> 00:11:29.420
I mean, you can parse it on as many levels as you want, but you know, you have your stoplight model, you know, having it rewrite sentences that are like, gosh, I keep saying this in the same way.
00:11:29.500 --> 00:11:30.940
What's another way to say this?
00:11:31.100 --> 00:11:32.860
And it'll rewrite it for you.
00:11:32.940 --> 00:11:35.259
That's another pretty low stakes.
00:11:35.899 --> 00:11:54.379
I think the next step up is probably like summarizing, taking bigger chunks of stuff and condensing it because it's still your own ideas, and you know, it's it's not necessarily taking over clinical judgment, it's not generating new information, it's just summarizing.
00:11:54.540 --> 00:12:00.779
So that's kind of like maybe like in where you would put the yellow on your stop line on the model.
00:12:01.100 --> 00:12:16.779
Well, you know, I was wondering because you talked about one of the ways that you interact with AI now is is doing the audio and just kind of almost a flow of thought, you know, of kind of how your thoughts are flowing, and you just kind of talk it out.
00:12:16.940 --> 00:12:20.300
Yes, and so and then have it summarize that.
00:12:20.540 --> 00:12:23.980
And I would imagine that could be a style of report writing.
00:12:24.139 --> 00:12:34.220
I don't know if I could do that at this stage yet, because I have 20-some odd years of in my head while I'm typing, kind of doing that kind of dialogue.
00:12:34.379 --> 00:12:46.779
But I imagine, especially newer people in the field, I do know that talking things through with other clinicians about particular cases certainly helps me crystallize things.
00:12:46.940 --> 00:13:02.700
And I imagine the AI can very effectively kind of crunch down all the stuff that maybe you've been meandering and thinking about to a more concise thing that that can lead to either you writing or you know, incorporating within the report process.
00:13:03.019 --> 00:13:10.540
I mean, I love AI as a as a thought partner, as someone to bounce ideas off or think about how would this read to a parent?
00:13:10.700 --> 00:13:14.300
You know, if you were a parent, how would this hit you?
00:13:14.379 --> 00:13:18.300
You know, and it'll be like, well, honestly, you know, this is pretty harsh.
00:13:18.460 --> 00:13:22.300
You know, and so in in that regard, I like I like the back and forth.
00:13:22.860 --> 00:13:39.500
Do you and I'm gonna I'm gonna deviate a little bit because I'm like, do you believe that its ability to mimic a particular persona is accurate with the actual personas that it's impersonating?
00:13:39.659 --> 00:13:45.420
So, for example, you mentioned if you were a parent, how would you picture this?
00:13:45.820 --> 00:14:04.540
Now, it gives you feedback that you think maybe a parent would say, but I've also seen some research that says, you know, because this is a little different area, because the research is generating AI-related data to come up with conclusions about individuals or humans, right?
00:14:04.700 --> 00:14:14.379
In other words, it creates fictitious personas and gives results, but they've actually compared that to real results and they find there's discrepancies.
00:14:14.460 --> 00:14:25.019
And so is it a mistake for us to start trusting it as uh, let's say, hey, I'm gonna give this to uh Kentucky school psychologists.
00:14:25.180 --> 00:14:27.820
Uh, what would be some relatable stuff to talk about?
00:14:27.899 --> 00:14:32.300
Or if you were a Kentucky school psychologist, how would you respond to these items?
00:14:32.620 --> 00:14:35.180
And it somehow leads me astray.
00:14:35.500 --> 00:14:45.259
I'm just kind of curious because I've seen some of that literature, at least from the research angle, that maybe says we probably shouldn't trust that type of use of AI.
00:14:45.500 --> 00:14:51.580
Well, to your first point, if you've met, you know, the same if you met one person with autism, you met one person with autism.
00:14:51.740 --> 00:14:53.740
If you met one parent, you met one parent.
00:14:53.899 --> 00:14:59.659
So, you know, is it going to react the way that a parent in a meeting is going to react?
00:14:59.820 --> 00:15:02.940
I mean, you could practice on another parent and it would be different.
00:15:03.100 --> 00:15:13.580
So, you know, there's no way to approximate or to, you know, determine how any one person's going to respond.
00:15:13.820 --> 00:15:22.540
I've seen research that suggests that AI is very good for psychometric purposes for basically substituting for human participants.
00:15:23.580 --> 00:15:30.220
Because it's trained on all of this on the internet, essentially, it does a pretty good job of approximating.
00:15:30.460 --> 00:15:42.460
You're always, I think you're always going to find one study that says one thing, one study that says another, and it's going to be depending on method and error, you know, just probabilistically speaking.
00:15:42.620 --> 00:15:46.300
So I think that it just depends.
00:15:46.379 --> 00:15:55.820
But I that practice is good, especially for a trainee, because part of that's getting over, you know, what I gotta tell this this parent that their child has a disability.
00:15:55.980 --> 00:16:00.060
And that that that's hard for a lot of people was for me.
00:16:00.379 --> 00:16:10.060
And just you know the exposure, even though it's you know, it's not it's not true exposure, it's a form of exposure, and I think that could be valuable.
00:16:10.379 --> 00:16:11.580
No, I I agree with you.
00:16:11.740 --> 00:16:16.379
I just uh sometimes want to play devil's advocate with our assumptions, right?
00:16:16.540 --> 00:16:23.259
Especially with AI, the moment you start assuming that this is an absolutely accurate rendering of fill in the blank.
00:16:23.580 --> 00:16:31.980
And I like that idea, you know, of being able to bounce ideas against somebody and give, you know, just tell the AI what type of background.
00:16:32.220 --> 00:16:43.980
You're an advertising executive, you're a marketing specialist, you're a special ad director, you know, whatever it might be, and to maybe lend yourself another viewpoint or perspective.
00:16:44.220 --> 00:16:48.379
Because you and I, from our psychological backgrounds, might think, oh, this is brilliant.
00:16:48.460 --> 00:16:49.899
This is a great way of viewing it.
00:16:49.980 --> 00:16:55.259
And then we ask about a parent's view, a teacher's view, and they're like, we don't understand this.
00:16:55.420 --> 00:16:57.899
Like, what is all this jargon you're throwing at us?
00:16:58.060 --> 00:17:06.059
And so it's valuable, but I didn't know because of concerns about hallucination and other sorts of things, you know, is that a way?
00:17:06.299 --> 00:17:15.659
Are we knowing more now on whether we need to we can still trust that other perspective, or is that perspective just really not based in reality at all?
00:17:15.980 --> 00:17:20.379
And it sounds like from what you're saying, you know, generally, yeah, for more generic purposes.
00:17:20.539 --> 00:17:32.539
Now, if I asked it to say, I've got to talk to Adam Lockwood and I need his opinions on things, I probably wouldn't trust the AI to be absolutely nail you down personality-wise.
00:17:32.619 --> 00:17:40.940
Or maybe it actually would be really good because you've used AI so much that it knows you to a T and can mimic your behaviors.
00:17:41.259 --> 00:17:54.220
Yeah, if you go and you look in um most of the large language models, if you click the memory button, that's one of the new things that that Chat GPT 5.6, which came out yesterday, people are listening to this later.
00:17:54.460 --> 00:18:00.779
You can click the memory button and it does this kind of like star wash and then it'll tell you everything about you.
00:18:00.859 --> 00:18:03.339
And it's it's spot on scary.
00:18:03.819 --> 00:18:07.179
But as far as trust goes, I wouldn't trust AI.
00:18:08.059 --> 00:18:10.779
No, I wouldn't say trust is the right word.
00:18:10.939 --> 00:18:13.980
Um I would not trust it.
00:18:14.139 --> 00:18:23.899
I would always act as an expert in the loop and and assume that whatever it's generating is going to have some hallucinations because it will.
00:18:24.139 --> 00:18:26.379
It's gonna make things make things up at times.
00:18:26.699 --> 00:18:34.859
Well, and I think that gets it back to our a good point regarding our AI kind of uh continuum of AI report writing.
00:18:35.259 --> 00:18:40.379
So we've gone through a couple of levels with grammar and edgeing, doing awkward language.
00:18:40.539 --> 00:18:45.659
We talked a bit about summarizations of interviews or thought processes.
00:18:45.819 --> 00:18:49.500
What would be next in your mind going up that kind of continuum?
00:18:50.459 --> 00:18:52.859
I think maybe recommendations.
00:18:53.419 --> 00:18:56.219
Again, it's at this point it's starting to synthesize more.
00:18:56.299 --> 00:18:58.219
It's taking things from other places.
00:18:58.379 --> 00:19:05.019
Um so and it's generating novel information, so it's a jump up.
00:19:05.259 --> 00:19:15.019
It is a jump up, but it's that uh generating of recommendations, drafting sections, uh, where you're actually kind of handing the keys over a little bit more.
00:19:15.179 --> 00:20:05.229
Um you know, you you still have a lot of you should still be looking at the output quite a bit, but um this is where you're you're getting into higher and a much more of a need for oversight and also the possibility of your own skills becoming rusty.
00:20:05.469 --> 00:20:05.789
Right.
00:20:06.029 --> 00:20:28.509
So this is the you're getting more at these higher ends on the I think of it as set it and forget it, like okay, here's the data, and I'm just gonna go and put it into the grinder of AI, and then it's gonna spit out this wonderful, you know, curated hamburger meat or whatever it is that I'm making, or sausage, or whatever.
00:20:28.669 --> 00:20:38.429
And I don't want to know other than when I plopped it in, it's just gonna spit out, and then I trust it innately, which I think we've discussed at ad nauseum.
00:20:38.509 --> 00:20:43.549
Like you can't trust the AI, you're gonna have to know with your own clinical judgment.
00:20:43.709 --> 00:20:54.669
And so, is there particular profiles of school psychologists who might be able to use it in this kind of manner and still do it in an ethical legal way?
00:20:54.989 --> 00:21:02.509
Or is there just no level where we wouldn't uh have to have some significant kind of hand in things?
00:21:03.229 --> 00:21:16.669
I well, first of all, I think I skipped over a level which is like help me regenerate sections, which I think is a little bit is like below that, where it's like, yes, you've really you really dove in there.
00:21:16.749 --> 00:21:18.269
You were really I almost said delve.
00:21:18.589 --> 00:21:24.189
Take me back to the the older AI days and delve was always part of the output.
00:21:24.989 --> 00:21:29.709
But you're I think that's a little less than just straight up generating.
00:21:29.869 --> 00:21:41.069
But the kind of person who I think could be okay with that is your more seasoned school psychologist who is able to look over output and be like, yes, this is this is correct.
00:21:41.389 --> 00:21:54.589
This is where I do not want interns, I don't want even quite honestly people in their first or second year personally, myself as a trainer, using AI for this because they need to build those skills before they hand it over.
00:21:54.749 --> 00:21:58.909
And I don't, you've probably heard my spiel is I always think of AI as an intern.
00:22:00.429 --> 00:22:06.109
And so it's a confident intern, and those are the worst kind of interns to have.
00:22:06.509 --> 00:22:18.749
You know, the interns who are anxious, you know, those ones are the best because they're like, I don't know, they're gonna double triple check, they're gonna come to you with questions, they're gonna say, I'm not sure.
00:22:18.909 --> 00:22:25.549
It's the one, you get that one every few years, and typically uh it's a he in my experience.
00:22:26.029 --> 00:22:32.989
He might have worked as a teacher or something that makes him think that I got this, whatever it is, maybe it's a personality.
00:22:33.229 --> 00:22:40.189
They're the ones that that you have to really watch out for because they're gonna be overly confident and they're gonna get outside their lane.
00:22:40.509 --> 00:22:45.869
And so if you think about it like that, I don't want an intern supervising an intern.
00:22:46.509 --> 00:22:52.509
I don't want a first-year school psychologist supervising, I don't want a second year supervising.
00:22:52.589 --> 00:22:59.389
And in fact, NASP has rules that you have to be practicing for a certain amount of time before you can be a supervisor.
00:22:59.629 --> 00:23:18.909
So I think if you think about it like that, those are the people who I don't want using AI for anything other than maybe rewording, grammar, things like that, unless they're using AI, they're writing first, and then having AI and comparing the two, I think there can be value in that.
00:23:19.149 --> 00:23:22.349
But those are the people who I don't want to write.
00:23:22.429 --> 00:23:29.469
The people who might be okay with it are the people who do this so much that they don't need to build that skill.
00:23:29.549 --> 00:23:39.869
But even those people, Dan, I worry we'll lose those skills if they have AI write reports each and every time for the entire report.
00:23:40.269 --> 00:23:56.189
I mean you're gonna have to come up with some sort of of rules, whether it's personal rules that you have or some guidance from NASP or VAPA that is, you know, do I write one section?
00:23:56.349 --> 00:23:58.349
Do I write one out of ten reports?
00:23:58.749 --> 00:24:01.869
Because otherwise you're gonna lose those skills.
00:24:02.269 --> 00:24:22.909
Yeah, I talk about that as uh what I call my foxtrot approach, which is you know, like quick, quick, slow, except for you know, interns, it's slow, slow, slow, slow, quick, you know, and it's a ratio, really, is what it gets at of how many do I do from AI, how many reports do I primarily do from scratch.
00:24:23.229 --> 00:24:29.229
And as you get more experience, the more you can do with AI, and the fewer you need to do with scratch.
00:24:29.389 --> 00:24:37.229
But I think, and I don't know if there's going to be a guidance that you can really have on it because it's almost a self-awareness.
00:24:37.389 --> 00:24:46.749
Like I know uh I've gone a little bit too far on a presentation or two relying on AI, and I'm up there presenting, and I do an okay job, right?
00:24:46.829 --> 00:24:52.429
I mean, it's not that I don't know the material, I don't know it at the level that I want to know it.
00:24:52.509 --> 00:24:58.029
And I've done this long enough when I really I'm just in sync with the material.
00:24:58.189 --> 00:24:59.389
Like I know this.
00:24:59.629 --> 00:25:01.869
People can throw whatever questions at me.
00:25:02.029 --> 00:25:05.549
I probably have the answer, I know the rationale behind it.
00:25:05.709 --> 00:25:14.829
And there were some points with a presentation or two, especially in an area that maybe I'm not as up to speed with in my background, that I overrelied on it.
00:25:14.909 --> 00:25:19.149
Uh, and I I didn't really fully realize it until I got in that situation.
00:25:19.309 --> 00:25:21.149
And then I've I've backpedaled, right?
00:25:21.229 --> 00:25:28.269
And I don't use it quite to that level uh for that type of purpose, in particular with topics I'm not as familiar with.
00:25:28.429 --> 00:25:30.829
And so I think report writing is going to be the same.
00:25:30.989 --> 00:25:43.309
You have to be uh aware of what you know and what you don't know, and a seasoned professional should know like, yeah, I know this kid, I've written this report, I know this data, and you know, I'm very confident.
00:25:43.549 --> 00:25:59.389
But then if you go back to a scratch report and you're beginning to forget, like, uh now what section did I put this in, you've probably gone too far in the reliance to AI, and you need to sit down and really carve out more time to do kind of those from scratch.
00:25:59.549 --> 00:26:12.509
But I can't tell you, you know, as an expert in AI, or and I'm sure you wouldn't either, if I take somebody and say I have 10 years of experience and I go, Oh, okay, you're at a five AIs to one scratch ratio.
00:26:12.909 --> 00:26:17.309
Like there's too much individual aspects to that.
00:26:17.469 --> 00:26:21.149
I think it's gonna be just up to the professional.
00:26:21.309 --> 00:26:22.429
And I totally agree.
00:26:22.509 --> 00:26:26.589
I don't up to the professional if you have four or more years experience.
00:26:26.749 --> 00:26:41.869
I think if you don't have at least four years of experience in the field, you shouldn't have the option of doing lots of AI because the problem is not only will you be de-skilled, you will be no skilled because you probably didn't have the full skills in place.
00:26:42.109 --> 00:26:44.349
NASP has acknowledged that, right?
00:26:44.429 --> 00:26:49.469
Because we have all those domains, they're impossible to cover in three years or two years.
00:26:49.629 --> 00:26:57.229
And they acknowledge like you are continuing to learn this process well into your career, your first few years of the career.
00:26:57.309 --> 00:27:00.989
And so with the AI report writing, you need to view it the same way.
00:27:01.149 --> 00:27:04.509
You are not an expert report writer the minute you graduate.
00:27:04.669 --> 00:27:12.589
You're much better than a typical grad student, but you're not to a point where you could confidently offload a lot of this to AI.
00:27:12.749 --> 00:27:15.869
At least that's kind of the way I visualize and I've explained it.
00:27:16.269 --> 00:27:27.709
Well, your your presentation example brings up another thing that I usually talk about when I present, and that is writing is not a passive endeavor.
00:27:29.069 --> 00:27:35.149
You know, it's partially documentation, but it's partially where we formalize uh ideas.
00:27:35.869 --> 00:27:42.749
It's through living in those data, it's through writing that we come to understand a student or a client or a patient.
00:27:42.989 --> 00:27:56.669
And if we are completely outsourcing that, my my thought is that we're not gonna know the needs and strengths of those people because we're just we're going through it too quickly.
00:27:56.829 --> 00:28:03.549
So yeah, it's that's I don't know where that balance lies, but I think it's something that people are gonna have to figure out.
00:28:04.349 --> 00:28:14.429
Yeah, I almost think of it as out of the loop in that in that model, you know, and that I have a historian friend who talks about living within the context.
00:28:14.749 --> 00:28:19.629
And I think particularly with our report, to like we need to live within the context of that data.
00:28:19.869 --> 00:28:29.789
And if we have an AI generate recommendations, I feel very confident if I've lived in that data to say whether those are good or bad personalized recommendations.
00:28:30.189 --> 00:28:38.509
If I haven't lived in that data and I've offloaded all these other aspects of the report, when it comes to recommendations, what am I going by?
00:28:38.589 --> 00:28:47.069
I'm going by my generic, uh, this is kind of like a second grader, yeah, this seems appropriate, but maybe not for that second grader, right?
00:28:47.149 --> 00:29:06.829
I'm relying on a nomethetic or kind of a norm-based thought process and not an ideographic, which is to your point with AI, ideographic or the individual is really good at helping with that because it can take all these factors in and make it much more specific to the individual that we are assessing.
00:29:07.149 --> 00:29:07.949
Absolutely.
00:29:08.189 --> 00:29:10.189
There's one more level, I think, Dan.
00:29:10.989 --> 00:29:13.949
Okay, let's do one more level and then we're about out of time on this.
00:29:15.149 --> 00:29:17.549
The last level is clinical judgment.
00:29:17.789 --> 00:29:31.549
That is, and it could cut we already see this in risk prediction, but essentially that AI will make the decision of whether someone's eligible or as a service.
00:29:32.109 --> 00:29:39.069
And to me, that is a level of risk um that to me is greater than even generating the report.
00:29:39.229 --> 00:29:44.909
Um we know that these things have bias baked into them, we know that they hallucinate.
00:29:45.149 --> 00:29:54.509
And so um I think it you can have AI to help you map out oh, here's some things that that look like ADHD.
00:29:54.669 --> 00:30:00.509
Here's some things that you know, using it in that way, kind of like a thought partner, can be helpful.
00:30:00.749 --> 00:30:10.349
But for me to have AI say eligible or not eligible, um is a step, is a level that I'm not comfortable going myself.
00:30:10.669 --> 00:30:28.989
So kind of like uh using AI maybe with a diagnostic suggestions, though that's a slippery slope because uh maybe it doesn't consider something that you would, and you just kind of go, well, the AI didn't come up with that, so I guess it's not going to be that, you know, depending on how good the AI is and what its suggestions are.
00:30:29.069 --> 00:30:42.189
But I'm thinking also to uh, you know, for the Basque, if you look at report of the Basque, it gives recommendations of like here are different diagnoses, here are symptoms that they might meet.
00:30:42.349 --> 00:30:46.429
And if I just went by that, it doesn't cover everything, right?
00:30:46.589 --> 00:30:50.829
It doesn't cover a lot of different disorders that are even childhood disorders.
00:30:50.989 --> 00:30:52.029
It gives some options.
00:30:52.189 --> 00:31:04.669
I appreciate its feedback, but I also know like this is not it's not going through the whole DSM, it's focusing on like five disorders, and it tells you how likely, and if it's unlikely, it just doesn't include it in the printout.
00:31:04.909 --> 00:31:24.349
And so I'm worried because that's a just a very basic understanding of how AI would be able to convey maybe the same options for you, and it's almost maybe better that you wait to have it give suggestions after you've struggled and come up with your own thought about what it might be.
00:31:24.509 --> 00:31:29.549
And then it may open your eyes to, oh, there are some other options I need to at least consider.
00:31:29.789 --> 00:31:32.589
But you've come to your own conclusion initially.
00:31:33.069 --> 00:31:42.189
So it's almost like the argument, do the work first, and then have AI come in and say, give me some other options that this could be given the data.
00:31:42.269 --> 00:31:46.509
And maybe in some cases you'll be like, oh, how could I have not seen that?
00:31:46.669 --> 00:31:59.469
Uh and I think that would be a better use of the AI report writer from a diagnostic sense, if that AI report writer even offers that, because there's a lot of report writers that are not going to go into that diagnostic.
00:31:59.629 --> 00:32:04.029
I think just liability-wise, you know, what if it makes the wrong diagnosis?
00:32:04.189 --> 00:32:07.709
Is it the clinician or will it be the AI that gets sued?
00:32:07.789 --> 00:32:10.989
So they don't want to open themselves up to that type of liability either.
00:32:11.069 --> 00:32:13.069
I would think it'd be part of it.
00:32:13.469 --> 00:32:19.149
I think the liability always falls on the practitioner, at least that's how I think of it.
00:32:19.469 --> 00:32:26.349
I think this is the importance, the take homes I have is if you use AI to write reports, know the data.
00:32:26.509 --> 00:32:28.029
Know the data very well.
00:32:28.269 --> 00:32:36.669
Do not blindly follow and keep your clinical judgment, and then you're probably gonna be um in a better place.
00:32:36.909 --> 00:32:41.309
And that's easier said than done because we're all so rushed.
00:32:41.469 --> 00:32:44.509
It's really easy to just want to be like, okay, do this for me.
00:32:44.589 --> 00:32:46.589
Great, it's done, move on.
00:32:46.989 --> 00:32:48.509
But I think that will get you into trouble.
00:32:48.909 --> 00:32:50.349
I totally agree with you, Adam.
00:32:50.509 --> 00:32:54.829
And I think that's a really good stopping point for this particular episode.
00:32:54.989 --> 00:33:00.109
And yet again, we've gotten through one little iota of the AI and report writing.
00:33:00.269 --> 00:33:06.429
So I have a feeling that we will have some future episodes, certainly diving into more detail.
00:33:06.669 --> 00:33:10.589
But uh, this is the end of our Beyond the Bot AI and mental health.
00:33:10.749 --> 00:33:14.349
Uh, hope you're enjoying these episodes and that you continue coming back.
00:33:14.509 --> 00:33:19.309
I know Adam and I have been enjoying chatting with one another about these issues, if nothing else.
00:33:19.469 --> 00:33:27.069
And hopefully you've gained some other insights that maybe go a little bit beyond just the trite little lists of things to consider.
00:33:27.469 --> 00:33:30.269
So uh we will see you for our next episode.
00:33:30.829 --> 00:33:31.549
Thanks.
00:33:31.789 --> 00:33:32.669
Thank you.
00:01:01.340 --> 00:01:04.299
The following is a site to practice production.
00:01:14.540 --> 00:01:18.620
Welcome to Beyond the Bot AI and Mental Health.
00:01:18.780 --> 00:01:28.620
I'm Dan Florel, a professor at Eastern Kentucky University, and we are going to be continuing on this episode to talk about AI report writing.
00:01:28.859 --> 00:01:48.859
If you recall from our last episode, we got into a little bit of the weeds, really into AI report recommendations and a little bit into how to format different AI reports that are from the same student or for the same client for different audiences and kind of focused on that.
00:01:49.020 --> 00:02:00.379
So this time around, we're going to be more focusing on kind of the levels of use that you could still argue are AI-assisted reports and what those exactly look like.
00:02:00.539 --> 00:02:05.740
But before we get into that, I want to introduce my co-host and partner in crime here.
00:02:06.379 --> 00:02:13.419
Adam, why don't you go ahead and introduce yourself and then we can get launched into more AI report writing discussion.
00:02:14.379 --> 00:02:15.259
Thanks, Dan.
00:02:15.419 --> 00:02:18.379
Adam Lockwood at Kent State University.
00:02:19.020 --> 00:02:21.740
Conflict of interest, same as before.
00:02:22.060 --> 00:02:26.300
Report writing, subject matter expert for uh for par.
00:02:28.140 --> 00:02:28.780
Yep.
00:02:30.140 --> 00:02:36.300
So we're gonna rely on people listening to uh our episodes sequentially as they build up.
00:02:36.620 --> 00:02:47.900
But you know, I think when we were talking about AI report writing, this is a huge issue, one that people continually talk to us about in our presentations.
00:02:48.140 --> 00:02:51.900
And fortunately, we are getting more knowledge about what those look like.
00:02:51.979 --> 00:02:55.900
And of course, there's a lot more products out there than ever before.
00:02:56.060 --> 00:03:17.419
And I just kind of go by if you've gone to the NAS convention and you go down to the vending area over the last three years, it's like they are exponentially more and more new report writers, and frankly, ones that I think are better every year, at least what they claim to be able to do, uh, versus what the first ones three years ago were claiming.
00:03:17.579 --> 00:03:18.859
So real impressive.
00:03:19.019 --> 00:03:28.780
But I think we need to get a little into the weeds, a little more general versus particular products, and that is exactly what are we considering AI assisted report writing?
00:03:28.939 --> 00:03:44.139
Adam, why don't you, because you run a lot of the research that we do in the field on AI, what would you consider kind of these different levels that we look at in regards to an AI assist for our reports?
00:03:44.460 --> 00:03:50.299
Well, I mean, honestly, we've been using AI and our report writing for quite some time.
00:03:50.859 --> 00:03:55.259
So at the most basic level, it's things like Grammarly.
00:03:55.500 --> 00:04:05.739
You know, technically, Grammarly is artificial intelligence and it's natural language processing, which is a similar technology to Chat GPT.
00:04:07.099 --> 00:04:19.980
One thing I always want to point out to people, and that a few people are caught off guard by, is that you should not be using Grammarly to help you with your reports unless you have an enterprise account.
00:04:20.860 --> 00:04:24.139
Because it trains on the data you provide it.
00:04:24.540 --> 00:04:34.780
So if you're using it just for grammar, spelling, and editing, it's taking your sensitive data and using it for who knows what.
00:04:35.019 --> 00:04:56.139
So that's the first level is this kind of grammarly, uh grammar spelling checking typos, and um that's pretty low stakes from a um clinical judgment, loss of skills uh perspective, but it can be high stakes from a data security, FERPA, HIPAA standpoint.
00:04:56.379 --> 00:05:01.980
So um that's that's kind of like the first level I think of when I think of AI.
00:05:02.860 --> 00:05:08.300
Yeah, so grammarly I get that, but what about like Microsoft Word, Google Docs?
00:05:08.540 --> 00:05:14.379
Because they do offer grammar, you know, punctuation spelling, you know, help.
00:05:14.860 --> 00:05:17.820
That's not really is that still considered AI?
00:05:17.900 --> 00:05:23.260
And would the same concerns apply when you're using those to write up the reports?
00:05:23.580 --> 00:05:27.340
That's gonna depend on a case-by-case basis.
00:05:27.580 --> 00:05:36.140
You know, spell check typically, my understanding, I'm not a computer programmer, but it's not using this process.
00:05:36.700 --> 00:05:47.340
It's also not underlying sentences and being like, this sentence is a fragment, or you know, consider rewording and giving you suggestions.
00:05:47.820 --> 00:05:50.780
Um so it's a little bit different.
00:05:50.939 --> 00:05:55.260
At this point, though, AI is inserted into everything, and you need to be careful.
00:05:55.420 --> 00:06:13.660
You know, if you use Google, there's a good chance that Gemini, which is generative AI, is integrated into your Google Docs, into your spreadsheets, into every single component.
00:06:13.820 --> 00:06:17.020
The same for Word, Copilot is up in everything.
00:06:17.260 --> 00:06:29.660
So if you're a Google workspace, a Google campus or a um Microsoft campus, you should definitely find out is do we have an enterprise account?
00:06:29.820 --> 00:06:32.939
And if so, what can I use it?
00:06:33.020 --> 00:06:38.060
Um, what's what's ask someone in IT or legal or administration?
00:06:38.300 --> 00:06:44.300
Is it okay for me to be using Gemini Integrated, or is it okay for me to be using Copilot?
00:06:44.540 --> 00:06:52.460
Well, I was gonna say because you know, back in the day, obviously Word and maybe some to lesser extent Google Docs, I mean it resided on your computer.
00:06:52.620 --> 00:06:57.020
You could store it there, and Microsoft kind of lost control of it for the most part afterwards.
00:06:57.100 --> 00:07:05.020
But now they've put everything into Microsoft 360, and of course, Google has always resided into these cloud servers.
00:07:05.260 --> 00:07:14.939
And then, of course, they've been integrating their premier product, whether it's co-pilot with Microsoft or Gemini with uh Google, into the products.
00:07:15.100 --> 00:07:17.740
Like there's really not even a selection process.
00:07:17.900 --> 00:07:33.340
So I guess from a most basic, most conservative view, one should always have the enterprise or business level kind of access that most organizations people work for would have.
00:07:33.500 --> 00:07:46.939
Uh, with the caveat, of course, being if you're a solo private practice, that you probably don't have that level of protection in an individual account, or if you're using the free account, certainly not.
00:07:47.180 --> 00:08:03.340
In which case, I guess you would argue, Adam, that uh maybe they shouldn't be using that or they should look into upgrading to a higher level where their materials aren't being used for training purposes or otherwise captured beyond what you're using it for.
00:08:03.580 --> 00:08:10.540
I've consulted with people in private practice to help them get BAAs in place for Jim and I co-pilot as well.
00:08:10.780 --> 00:08:17.420
It's a pain, but it's definitely doable, and it's something that I would do if I was in private practice for sure.
00:08:18.060 --> 00:08:18.379
Okay.
00:08:18.860 --> 00:08:20.140
And is that something?
00:08:20.300 --> 00:08:20.939
What do you do?
00:08:21.100 --> 00:08:26.780
Ask AI kind of how to go about getting a BAA with those organizations or or services?
00:08:27.180 --> 00:08:32.620
It's been almost a year since the last time I did it, and it always changes on how you do it.
00:08:32.860 --> 00:08:54.059
But um essentially, you know, you get an enterprise account, um, you pay for a certain level of um, you know, a suite that is where they have a business associates agreement with you, they email you a document that says the BAA is in place, and um you know, depending on your level of risk tolerance.
00:08:54.139 --> 00:09:00.059
For me, I think if I had that BAA an email, I would I would be okay, you know, I would be comfortable.
00:09:01.820 --> 00:09:02.059
Right.
00:09:02.460 --> 00:09:10.860
If I had something from Goo, I I always encourage people get an email with the BAA saying that it's HIPAA compliant.
00:09:11.100 --> 00:09:14.700
I'm not a lawyer, so you know, I always give that copy out.
00:09:14.779 --> 00:09:37.180
But I for my own level of comfort, I want something from the company if I'm in private practice, or I want some something from someone in authority in my organization, whether that's IT legal or administration saying, yes, you can use you know this when you're working with students or when you're using this with clients.
00:09:37.980 --> 00:09:39.899
And I'd say that email.
00:09:40.700 --> 00:09:41.420
Yeah, yeah.
00:09:41.580 --> 00:09:44.379
I was gonna say that's one of those things you want documented.
00:09:44.460 --> 00:09:47.899
That's one also the advantage if you're working for a larger organization.
00:09:48.060 --> 00:09:55.180
So, for example, most school districts that have Google that are a Google campus, they'll probably have the enterprise as part of it.
00:09:55.340 --> 00:09:59.580
And what I've heard anyway is that that makes it FERPA compliant.
00:09:59.659 --> 00:10:08.700
Um, I haven't heard about HIPAA compliant, but in the schools, of course, the FERPA law is going to be the one that you would be more attended to anyway.
00:10:09.180 --> 00:10:17.659
I think we've given some people the things to think about in regards to the level of security, regardless of how innocuous the AI seems.
00:10:17.820 --> 00:10:24.940
It's kind of like it's seeped itself into areas that you hadn't previously had to worry about in the same way.
00:10:25.340 --> 00:10:30.540
So at the lowest level, you were talking about grammar, spelling, editing, suggestions.
00:10:30.700 --> 00:10:33.820
It's kind of your low-level AI-assisted report, right?
00:10:34.060 --> 00:10:49.899
Or what kind of levels come next in regards to kind of the gradient of relatively benign, basic writing help to, of course, at the very end would be writing the whole thing, giving a diagnosis and going on kind of thing.
00:10:50.060 --> 00:10:55.340
What are some of the things in between that people might start considering uh adopting?
00:10:55.500 --> 00:10:59.340
And as you know, they're beginning to maybe embrace AI.
00:10:59.580 --> 00:11:16.540
I always tell people, kind of, let's go slowly, uh, you know, gradually integrate it within your practice to a point where you feel like you've either lost some of the specialization and skills that you need yourself, uh, and also where you're comfortable ethically, legally from that standpoint.
00:11:16.860 --> 00:11:29.420
I mean, you can parse it on as many levels as you want, but you know, you have your stoplight model, you know, having it rewrite sentences that are like, gosh, I keep saying this in the same way.
00:11:29.500 --> 00:11:30.940
What's another way to say this?
00:11:31.100 --> 00:11:32.860
And it'll rewrite it for you.
00:11:32.940 --> 00:11:35.259
That's another pretty low stakes.
00:11:35.899 --> 00:11:54.379
I think the next step up is probably like summarizing, taking bigger chunks of stuff and condensing it because it's still your own ideas, and you know, it's it's not necessarily taking over clinical judgment, it's not generating new information, it's just summarizing.
00:11:54.540 --> 00:12:00.779
So that's kind of like maybe like in where you would put the yellow on your stop line on the model.
00:12:01.100 --> 00:12:16.779
Well, you know, I was wondering because you talked about one of the ways that you interact with AI now is is doing the audio and just kind of almost a flow of thought, you know, of kind of how your thoughts are flowing, and you just kind of talk it out.
00:12:16.940 --> 00:12:20.300
Yes, and so and then have it summarize that.
00:12:20.540 --> 00:12:23.980
And I would imagine that could be a style of report writing.
00:12:24.139 --> 00:12:34.220
I don't know if I could do that at this stage yet, because I have 20-some odd years of in my head while I'm typing, kind of doing that kind of dialogue.
00:12:34.379 --> 00:12:46.779
But I imagine, especially newer people in the field, I do know that talking things through with other clinicians about particular cases certainly helps me crystallize things.
00:12:46.940 --> 00:13:02.700
And I imagine the AI can very effectively kind of crunch down all the stuff that maybe you've been meandering and thinking about to a more concise thing that that can lead to either you writing or you know, incorporating within the report process.
00:13:03.019 --> 00:13:10.540
I mean, I love AI as a as a thought partner, as someone to bounce ideas off or think about how would this read to a parent?
00:13:10.700 --> 00:13:14.300
You know, if you were a parent, how would this hit you?
00:13:14.379 --> 00:13:18.300
You know, and it'll be like, well, honestly, you know, this is pretty harsh.
00:13:18.460 --> 00:13:22.300
You know, and so in in that regard, I like I like the back and forth.
00:13:22.860 --> 00:13:39.500
Do you and I'm gonna I'm gonna deviate a little bit because I'm like, do you believe that its ability to mimic a particular persona is accurate with the actual personas that it's impersonating?
00:13:39.659 --> 00:13:45.420
So, for example, you mentioned if you were a parent, how would you picture this?
00:13:45.820 --> 00:14:04.540
Now, it gives you feedback that you think maybe a parent would say, but I've also seen some research that says, you know, because this is a little different area, because the research is generating AI-related data to come up with conclusions about individuals or humans, right?
00:14:04.700 --> 00:14:14.379
In other words, it creates fictitious personas and gives results, but they've actually compared that to real results and they find there's discrepancies.
00:14:14.460 --> 00:14:25.019
And so is it a mistake for us to start trusting it as uh, let's say, hey, I'm gonna give this to uh Kentucky school psychologists.
00:14:25.180 --> 00:14:27.820
Uh, what would be some relatable stuff to talk about?
00:14:27.899 --> 00:14:32.300
Or if you were a Kentucky school psychologist, how would you respond to these items?
00:14:32.620 --> 00:14:35.180
And it somehow leads me astray.
00:14:35.500 --> 00:14:45.259
I'm just kind of curious because I've seen some of that literature, at least from the research angle, that maybe says we probably shouldn't trust that type of use of AI.
00:14:45.500 --> 00:14:51.580
Well, to your first point, if you've met, you know, the same if you met one person with autism, you met one person with autism.
00:14:51.740 --> 00:14:53.740
If you met one parent, you met one parent.
00:14:53.899 --> 00:14:59.659
So, you know, is it going to react the way that a parent in a meeting is going to react?
00:14:59.820 --> 00:15:02.940
I mean, you could practice on another parent and it would be different.
00:15:03.100 --> 00:15:13.580
So, you know, there's no way to approximate or to, you know, determine how any one person's going to respond.
00:15:13.820 --> 00:15:22.540
I've seen research that suggests that AI is very good for psychometric purposes for basically substituting for human participants.
00:15:23.580 --> 00:15:30.220
Because it's trained on all of this on the internet, essentially, it does a pretty good job of approximating.
00:15:30.460 --> 00:15:42.460
You're always, I think you're always going to find one study that says one thing, one study that says another, and it's going to be depending on method and error, you know, just probabilistically speaking.
00:15:42.620 --> 00:15:46.300
So I think that it just depends.
00:15:46.379 --> 00:15:55.820
But I that practice is good, especially for a trainee, because part of that's getting over, you know, what I gotta tell this this parent that their child has a disability.
00:15:55.980 --> 00:16:00.060
And that that that's hard for a lot of people was for me.
00:16:00.379 --> 00:16:10.060
And just you know the exposure, even though it's you know, it's not it's not true exposure, it's a form of exposure, and I think that could be valuable.
00:16:10.379 --> 00:16:11.580
No, I I agree with you.
00:16:11.740 --> 00:16:16.379
I just uh sometimes want to play devil's advocate with our assumptions, right?
00:16:16.540 --> 00:16:23.259
Especially with AI, the moment you start assuming that this is an absolutely accurate rendering of fill in the blank.
00:16:23.580 --> 00:16:31.980
And I like that idea, you know, of being able to bounce ideas against somebody and give, you know, just tell the AI what type of background.
00:16:32.220 --> 00:16:43.980
You're an advertising executive, you're a marketing specialist, you're a special ad director, you know, whatever it might be, and to maybe lend yourself another viewpoint or perspective.
00:16:44.220 --> 00:16:48.379
Because you and I, from our psychological backgrounds, might think, oh, this is brilliant.
00:16:48.460 --> 00:16:49.899
This is a great way of viewing it.
00:16:49.980 --> 00:16:55.259
And then we ask about a parent's view, a teacher's view, and they're like, we don't understand this.
00:16:55.420 --> 00:16:57.899
Like, what is all this jargon you're throwing at us?
00:16:58.060 --> 00:17:06.059
And so it's valuable, but I didn't know because of concerns about hallucination and other sorts of things, you know, is that a way?
00:17:06.299 --> 00:17:15.659
Are we knowing more now on whether we need to we can still trust that other perspective, or is that perspective just really not based in reality at all?
00:17:15.980 --> 00:17:20.379
And it sounds like from what you're saying, you know, generally, yeah, for more generic purposes.
00:17:20.539 --> 00:17:32.539
Now, if I asked it to say, I've got to talk to Adam Lockwood and I need his opinions on things, I probably wouldn't trust the AI to be absolutely nail you down personality-wise.
00:17:32.619 --> 00:17:40.940
Or maybe it actually would be really good because you've used AI so much that it knows you to a T and can mimic your behaviors.
00:17:41.259 --> 00:17:54.220
Yeah, if you go and you look in um most of the large language models, if you click the memory button, that's one of the new things that that Chat GPT 5.6, which came out yesterday, people are listening to this later.
00:17:54.460 --> 00:18:00.779
You can click the memory button and it does this kind of like star wash and then it'll tell you everything about you.
00:18:00.859 --> 00:18:03.339
And it's it's spot on scary.
00:18:03.819 --> 00:18:07.179
But as far as trust goes, I wouldn't trust AI.
00:18:08.059 --> 00:18:10.779
No, I wouldn't say trust is the right word.
00:18:10.939 --> 00:18:13.980
Um I would not trust it.
00:18:14.139 --> 00:18:23.899
I would always act as an expert in the loop and and assume that whatever it's generating is going to have some hallucinations because it will.
00:18:24.139 --> 00:18:26.379
It's gonna make things make things up at times.
00:18:26.699 --> 00:18:34.859
Well, and I think that gets it back to our a good point regarding our AI kind of uh continuum of AI report writing.
00:18:35.259 --> 00:18:40.379
So we've gone through a couple of levels with grammar and edgeing, doing awkward language.
00:18:40.539 --> 00:18:45.659
We talked a bit about summarizations of interviews or thought processes.
00:18:45.819 --> 00:18:49.500
What would be next in your mind going up that kind of continuum?
00:18:50.459 --> 00:18:52.859
I think maybe recommendations.
00:18:53.419 --> 00:18:56.219
Again, it's at this point it's starting to synthesize more.
00:18:56.299 --> 00:18:58.219
It's taking things from other places.
00:18:58.379 --> 00:19:05.019
Um so and it's generating novel information, so it's a jump up.
00:19:05.259 --> 00:19:15.019
It is a jump up, but it's that uh generating of recommendations, drafting sections, uh, where you're actually kind of handing the keys over a little bit more.
00:19:15.179 --> 00:20:05.229
Um you know, you you still have a lot of you should still be looking at the output quite a bit, but um this is where you're you're getting into higher and a much more of a need for oversight and also the possibility of your own skills becoming rusty.
00:20:05.469 --> 00:20:05.789
Right.
00:20:06.029 --> 00:20:28.509
So this is the you're getting more at these higher ends on the I think of it as set it and forget it, like okay, here's the data, and I'm just gonna go and put it into the grinder of AI, and then it's gonna spit out this wonderful, you know, curated hamburger meat or whatever it is that I'm making, or sausage, or whatever.
00:20:28.669 --> 00:20:38.429
And I don't want to know other than when I plopped it in, it's just gonna spit out, and then I trust it innately, which I think we've discussed at ad nauseum.
00:20:38.509 --> 00:20:43.549
Like you can't trust the AI, you're gonna have to know with your own clinical judgment.
00:20:43.709 --> 00:20:54.669
And so, is there particular profiles of school psychologists who might be able to use it in this kind of manner and still do it in an ethical legal way?
00:20:54.989 --> 00:21:02.509
Or is there just no level where we wouldn't uh have to have some significant kind of hand in things?
00:21:03.229 --> 00:21:16.669
I well, first of all, I think I skipped over a level which is like help me regenerate sections, which I think is a little bit is like below that, where it's like, yes, you've really you really dove in there.
00:21:16.749 --> 00:21:18.269
You were really I almost said delve.
00:21:18.589 --> 00:21:24.189
Take me back to the the older AI days and delve was always part of the output.
00:21:24.989 --> 00:21:29.709
But you're I think that's a little less than just straight up generating.
00:21:29.869 --> 00:21:41.069
But the kind of person who I think could be okay with that is your more seasoned school psychologist who is able to look over output and be like, yes, this is this is correct.
00:21:41.389 --> 00:21:54.589
This is where I do not want interns, I don't want even quite honestly people in their first or second year personally, myself as a trainer, using AI for this because they need to build those skills before they hand it over.
00:21:54.749 --> 00:21:58.909
And I don't, you've probably heard my spiel is I always think of AI as an intern.
00:22:00.429 --> 00:22:06.109
And so it's a confident intern, and those are the worst kind of interns to have.
00:22:06.509 --> 00:22:18.749
You know, the interns who are anxious, you know, those ones are the best because they're like, I don't know, they're gonna double triple check, they're gonna come to you with questions, they're gonna say, I'm not sure.
00:22:18.909 --> 00:22:25.549
It's the one, you get that one every few years, and typically uh it's a he in my experience.
00:22:26.029 --> 00:22:32.989
He might have worked as a teacher or something that makes him think that I got this, whatever it is, maybe it's a personality.
00:22:33.229 --> 00:22:40.189
They're the ones that that you have to really watch out for because they're gonna be overly confident and they're gonna get outside their lane.
00:22:40.509 --> 00:22:45.869
And so if you think about it like that, I don't want an intern supervising an intern.
00:22:46.509 --> 00:22:52.509
I don't want a first-year school psychologist supervising, I don't want a second year supervising.
00:22:52.589 --> 00:22:59.389
And in fact, NASP has rules that you have to be practicing for a certain amount of time before you can be a supervisor.
00:22:59.629 --> 00:23:18.909
So I think if you think about it like that, those are the people who I don't want using AI for anything other than maybe rewording, grammar, things like that, unless they're using AI, they're writing first, and then having AI and comparing the two, I think there can be value in that.
00:23:19.149 --> 00:23:22.349
But those are the people who I don't want to write.
00:23:22.429 --> 00:23:29.469
The people who might be okay with it are the people who do this so much that they don't need to build that skill.
00:23:29.549 --> 00:23:39.869
But even those people, Dan, I worry we'll lose those skills if they have AI write reports each and every time for the entire report.
00:23:40.269 --> 00:23:56.189
I mean you're gonna have to come up with some sort of of rules, whether it's personal rules that you have or some guidance from NASP or VAPA that is, you know, do I write one section?
00:23:56.349 --> 00:23:58.349
Do I write one out of ten reports?
00:23:58.749 --> 00:24:01.869
Because otherwise you're gonna lose those skills.
00:24:02.269 --> 00:24:22.909
Yeah, I talk about that as uh what I call my foxtrot approach, which is you know, like quick, quick, slow, except for you know, interns, it's slow, slow, slow, slow, quick, you know, and it's a ratio, really, is what it gets at of how many do I do from AI, how many reports do I primarily do from scratch.
00:24:23.229 --> 00:24:29.229
And as you get more experience, the more you can do with AI, and the fewer you need to do with scratch.
00:24:29.389 --> 00:24:37.229
But I think, and I don't know if there's going to be a guidance that you can really have on it because it's almost a self-awareness.
00:24:37.389 --> 00:24:46.749
Like I know uh I've gone a little bit too far on a presentation or two relying on AI, and I'm up there presenting, and I do an okay job, right?
00:24:46.829 --> 00:24:52.429
I mean, it's not that I don't know the material, I don't know it at the level that I want to know it.
00:24:52.509 --> 00:24:58.029
And I've done this long enough when I really I'm just in sync with the material.
00:24:58.189 --> 00:24:59.389
Like I know this.
00:24:59.629 --> 00:25:01.869
People can throw whatever questions at me.
00:25:02.029 --> 00:25:05.549
I probably have the answer, I know the rationale behind it.
00:25:05.709 --> 00:25:14.829
And there were some points with a presentation or two, especially in an area that maybe I'm not as up to speed with in my background, that I overrelied on it.
00:25:14.909 --> 00:25:19.149
Uh, and I I didn't really fully realize it until I got in that situation.
00:25:19.309 --> 00:25:21.149
And then I've I've backpedaled, right?
00:25:21.229 --> 00:25:28.269
And I don't use it quite to that level uh for that type of purpose, in particular with topics I'm not as familiar with.
00:25:28.429 --> 00:25:30.829
And so I think report writing is going to be the same.
00:25:30.989 --> 00:25:43.309
You have to be uh aware of what you know and what you don't know, and a seasoned professional should know like, yeah, I know this kid, I've written this report, I know this data, and you know, I'm very confident.
00:25:43.549 --> 00:25:59.389
But then if you go back to a scratch report and you're beginning to forget, like, uh now what section did I put this in, you've probably gone too far in the reliance to AI, and you need to sit down and really carve out more time to do kind of those from scratch.
00:25:59.549 --> 00:26:12.509
But I can't tell you, you know, as an expert in AI, or and I'm sure you wouldn't either, if I take somebody and say I have 10 years of experience and I go, Oh, okay, you're at a five AIs to one scratch ratio.
00:26:12.909 --> 00:26:17.309
Like there's too much individual aspects to that.
00:26:17.469 --> 00:26:21.149
I think it's gonna be just up to the professional.
00:26:21.309 --> 00:26:22.429
And I totally agree.
00:26:22.509 --> 00:26:26.589
I don't up to the professional if you have four or more years experience.
00:26:26.749 --> 00:26:41.869
I think if you don't have at least four years of experience in the field, you shouldn't have the option of doing lots of AI because the problem is not only will you be de-skilled, you will be no skilled because you probably didn't have the full skills in place.
00:26:42.109 --> 00:26:44.349
NASP has acknowledged that, right?
00:26:44.429 --> 00:26:49.469
Because we have all those domains, they're impossible to cover in three years or two years.
00:26:49.629 --> 00:26:57.229
And they acknowledge like you are continuing to learn this process well into your career, your first few years of the career.
00:26:57.309 --> 00:27:00.989
And so with the AI report writing, you need to view it the same way.
00:27:01.149 --> 00:27:04.509
You are not an expert report writer the minute you graduate.
00:27:04.669 --> 00:27:12.589
You're much better than a typical grad student, but you're not to a point where you could confidently offload a lot of this to AI.
00:27:12.749 --> 00:27:15.869
At least that's kind of the way I visualize and I've explained it.
00:27:16.269 --> 00:27:27.709
Well, your your presentation example brings up another thing that I usually talk about when I present, and that is writing is not a passive endeavor.
00:27:29.069 --> 00:27:35.149
You know, it's partially documentation, but it's partially where we formalize uh ideas.
00:27:35.869 --> 00:27:42.749
It's through living in those data, it's through writing that we come to understand a student or a client or a patient.
00:27:42.989 --> 00:27:56.669
And if we are completely outsourcing that, my my thought is that we're not gonna know the needs and strengths of those people because we're just we're going through it too quickly.
00:27:56.829 --> 00:28:03.549
So yeah, it's that's I don't know where that balance lies, but I think it's something that people are gonna have to figure out.
00:28:04.349 --> 00:28:14.429
Yeah, I almost think of it as out of the loop in that in that model, you know, and that I have a historian friend who talks about living within the context.
00:28:14.749 --> 00:28:19.629
And I think particularly with our report, to like we need to live within the context of that data.
00:28:19.869 --> 00:28:29.789
And if we have an AI generate recommendations, I feel very confident if I've lived in that data to say whether those are good or bad personalized recommendations.
00:28:30.189 --> 00:28:38.509
If I haven't lived in that data and I've offloaded all these other aspects of the report, when it comes to recommendations, what am I going by?
00:28:38.589 --> 00:28:47.069
I'm going by my generic, uh, this is kind of like a second grader, yeah, this seems appropriate, but maybe not for that second grader, right?
00:28:47.149 --> 00:29:06.829
I'm relying on a nomethetic or kind of a norm-based thought process and not an ideographic, which is to your point with AI, ideographic or the individual is really good at helping with that because it can take all these factors in and make it much more specific to the individual that we are assessing.
00:29:07.149 --> 00:29:07.949
Absolutely.
00:29:08.189 --> 00:29:10.189
There's one more level, I think, Dan.
00:29:10.989 --> 00:29:13.949
Okay, let's do one more level and then we're about out of time on this.
00:29:15.149 --> 00:29:17.549
The last level is clinical judgment.
00:29:17.789 --> 00:29:31.549
That is, and it could cut we already see this in risk prediction, but essentially that AI will make the decision of whether someone's eligible or as a service.
00:29:32.109 --> 00:29:39.069
And to me, that is a level of risk um that to me is greater than even generating the report.
00:29:39.229 --> 00:29:44.909
Um we know that these things have bias baked into them, we know that they hallucinate.
00:29:45.149 --> 00:29:54.509
And so um I think it you can have AI to help you map out oh, here's some things that that look like ADHD.
00:29:54.669 --> 00:30:00.509
Here's some things that you know, using it in that way, kind of like a thought partner, can be helpful.
00:30:00.749 --> 00:30:10.349
But for me to have AI say eligible or not eligible, um is a step, is a level that I'm not comfortable going myself.
00:30:10.669 --> 00:30:28.989
So kind of like uh using AI maybe with a diagnostic suggestions, though that's a slippery slope because uh maybe it doesn't consider something that you would, and you just kind of go, well, the AI didn't come up with that, so I guess it's not going to be that, you know, depending on how good the AI is and what its suggestions are.
00:30:29.069 --> 00:30:42.189
But I'm thinking also to uh, you know, for the Basque, if you look at report of the Basque, it gives recommendations of like here are different diagnoses, here are symptoms that they might meet.
00:30:42.349 --> 00:30:46.429
And if I just went by that, it doesn't cover everything, right?
00:30:46.589 --> 00:30:50.829
It doesn't cover a lot of different disorders that are even childhood disorders.
00:30:50.989 --> 00:30:52.029
It gives some options.
00:30:52.189 --> 00:31:04.669
I appreciate its feedback, but I also know like this is not it's not going through the whole DSM, it's focusing on like five disorders, and it tells you how likely, and if it's unlikely, it just doesn't include it in the printout.
00:31:04.909 --> 00:31:24.349
And so I'm worried because that's a just a very basic understanding of how AI would be able to convey maybe the same options for you, and it's almost maybe better that you wait to have it give suggestions after you've struggled and come up with your own thought about what it might be.
00:31:24.509 --> 00:31:29.549
And then it may open your eyes to, oh, there are some other options I need to at least consider.
00:31:29.789 --> 00:31:32.589
But you've come to your own conclusion initially.
00:31:33.069 --> 00:31:42.189
So it's almost like the argument, do the work first, and then have AI come in and say, give me some other options that this could be given the data.
00:31:42.269 --> 00:31:46.509
And maybe in some cases you'll be like, oh, how could I have not seen that?
00:31:46.669 --> 00:31:59.469
Uh and I think that would be a better use of the AI report writer from a diagnostic sense, if that AI report writer even offers that, because there's a lot of report writers that are not going to go into that diagnostic.
00:31:59.629 --> 00:32:04.029
I think just liability-wise, you know, what if it makes the wrong diagnosis?
00:32:04.189 --> 00:32:07.709
Is it the clinician or will it be the AI that gets sued?
00:32:07.789 --> 00:32:10.989
So they don't want to open themselves up to that type of liability either.
00:32:11.069 --> 00:32:13.069
I would think it'd be part of it.
00:32:13.469 --> 00:32:19.149
I think the liability always falls on the practitioner, at least that's how I think of it.
00:32:19.469 --> 00:32:26.349
I think this is the importance, the take homes I have is if you use AI to write reports, know the data.
00:32:26.509 --> 00:32:28.029
Know the data very well.
00:32:28.269 --> 00:32:36.669
Do not blindly follow and keep your clinical judgment, and then you're probably gonna be um in a better place.
00:32:36.909 --> 00:32:41.309
And that's easier said than done because we're all so rushed.
00:32:41.469 --> 00:32:44.509
It's really easy to just want to be like, okay, do this for me.
00:32:44.589 --> 00:32:46.589
Great, it's done, move on.
00:32:46.989 --> 00:32:48.509
But I think that will get you into trouble.
00:32:48.909 --> 00:32:50.349
I totally agree with you, Adam.
00:32:50.509 --> 00:32:54.829
And I think that's a really good stopping point for this particular episode.
00:32:54.989 --> 00:33:00.109
And yet again, we've gotten through one little iota of the AI and report writing.
00:33:00.269 --> 00:33:06.429
So I have a feeling that we will have some future episodes, certainly diving into more detail.
00:33:06.669 --> 00:33:10.589
But uh, this is the end of our Beyond the Bot AI and mental health.
00:33:10.749 --> 00:33:14.349
Uh, hope you're enjoying these episodes and that you continue coming back.
00:33:14.509 --> 00:33:19.309
I know Adam and I have been enjoying chatting with one another about these issues, if nothing else.
00:33:19.469 --> 00:33:27.069
And hopefully you've gained some other insights that maybe go a little bit beyond just the trite little lists of things to consider.
00:33:27.469 --> 00:33:30.269
So uh we will see you for our next episode.
00:33:30.829 --> 00:33:31.549
Thanks.
00:33:31.789 --> 00:33:32.669
Thank you.
