WEBVTT
00:00:04.080 --> 00:00:15.279
Welcome to the Brian Fultis Behavioral Finance Podcast, where we unravel the mysteries of behavioral finance and unlock the secrets to making smarter, more informed decisions with your money.
00:00:15.519 --> 00:00:17.359
Now, here's your host, Dr.
00:00:17.440 --> 00:00:18.719
Brian Fultis.
00:00:24.879 --> 00:00:26.320
Welcome back, everybody.
00:00:26.480 --> 00:00:33.600
We're going to continue our conversation on using technology and how much should we trust it?
00:00:33.920 --> 00:00:36.960
Or should we never trust it at all?
00:00:37.280 --> 00:00:46.799
And that's where we want to wind up here having a good answer and having an actual framework on where we fit on this continuum by the end of this episode.
00:00:47.039 --> 00:01:15.040
And this is part two of the series on automation bias, which is putting too much trust in technology, to today's topic of algorithm inversion, which means when an algorithm gives you bad information, you have a very strong negative reaction to that, much stronger than if a human were to give you bad information.
00:01:15.359 --> 00:01:18.400
And so that's what we're going to be talking about today.
00:01:18.640 --> 00:01:28.799
So remember, as we said, automation bias, we were looking at the proceedings and this summary of automation bias using pilots back in the 90s.
00:01:29.040 --> 00:01:48.239
And we found that the two types of errors from the automation bias, omission, so not getting information from the cockpit or the computer, and also commission, where it gives you the wrong bit of information and you file or follow that blindly without checking your own information.
00:01:48.400 --> 00:01:52.959
Those are the two types of errors that lead to this automation bias.
00:01:53.760 --> 00:01:58.640
Same authors that I referred to last week, Kathleen Mosier and Linda Skitka.
00:01:58.799 --> 00:02:08.960
They both uh additionally, I made some extra notes that if you add a second person, that that does not necessarily help you alleviate it.
00:02:09.039 --> 00:02:23.199
It just basically gives you another excuse not to be vigilant and to just trust what the computer tells you, even though you would think another set of eyes would reduce that quite significantly.
00:02:23.520 --> 00:02:37.120
They also found that uh teaching about this automation bias actually does reduce people's bias and they actually will question and be more vigilant.
00:02:37.280 --> 00:02:56.479
So it's actually very encouraging for us because as we are learning about this, we're also learning that with awareness comes that ability to say that, hey, look, I'm using my type one thinking, my automatic way of thinking, maybe a little bit too much.
00:02:56.639 --> 00:03:02.719
Maybe it's time for me to really critically think or me be more critical in this particular area.
00:03:02.879 --> 00:03:17.120
And it's through that awareness that we can actually get to that spot where we can step in, be vigilant, or double check that the information that we are receiving from technology is actually correct.
00:03:17.919 --> 00:03:25.039
And so when we're talking about the messaging here, we have to realize that it is not about AI.
00:03:25.599 --> 00:03:28.400
Um it's a lot about our ego.
00:03:28.719 --> 00:03:36.159
And so we are way more willing to forgive other humans than we are machines.
00:03:36.960 --> 00:03:47.120
And I had this happen in my research work this summer, where I was writing, I'm writing this paper on uh withdrawal rate rules.
00:03:47.280 --> 00:03:53.280
And you're gonna hear a whole series on what I found this summer because I'm just on the back end of it.
00:03:53.439 --> 00:04:07.680
But I remember one of the days I was actually hadn't read this paper for a little while and just wanted to go back in and was asking ChatGPT to give me that summary of their methodology and what they were doing, because I just needed a refresher.
00:04:07.759 --> 00:04:12.240
I didn't want to go and find the paper and dig in and go through it all myself.
00:04:12.319 --> 00:04:14.159
So I was kind of leaning on them.
00:04:14.479 --> 00:04:34.800
And ChatGPT came back with a very complete, non-complete answer saying that this study actually went and did a a Monte Carlo Carlo simulation and that it was different from all the other papers.
00:04:35.360 --> 00:04:42.399
And it's through this the findings and then just knocked out like these bullet points and was fairly believable.
00:04:42.480 --> 00:04:54.560
You go, okay, well, that makes sense that um it may that somebody went to a Monte Carlo simulation with this data and that they would use it to basically recheck these findings.
00:04:54.879 --> 00:04:59.279
Until I had to step in and go, I don't think that's true.
00:04:59.439 --> 00:05:04.480
That sounds way off from the paper that I read a while back.
00:05:04.879 --> 00:05:16.319
Well, sure enough, I look at that paper and go, that's not even close to what they use previous empirical data, like like historical returns in their analysis.
00:05:16.639 --> 00:05:23.839
And not even one single mention, let alone their main finding around this Monte Carlo simulation.
00:05:24.000 --> 00:05:44.879
And I go, I felt so betrayed by AI, and I was really really angry about it, going, my reaction was way stronger than somebody saying, Hey, look, I think it's this, or or just basically lying, or whatever.
00:05:45.040 --> 00:05:48.800
If a human were to tell me, oh wait, I had that confused with something else.
00:05:48.879 --> 00:05:49.680
I'm sorry.
00:05:54.319 --> 00:06:05.759
You, but the way that AI presented that to me and and just gave it as this gospel, and I was like, I felt really betrayed by this.
00:06:05.920 --> 00:06:08.079
So what we're talking about here, that's what started.
00:06:08.160 --> 00:06:16.480
I go, man, I really had a strong negative response to this, saying, I'm never gonna have ChatGPT help me with any of these summaries.
00:06:16.560 --> 00:06:18.959
I gotta do it all myself, just like the old days.
00:06:19.120 --> 00:06:23.839
And it's like, okay, are you are you overreacting maybe a little bit?
00:06:24.160 --> 00:06:28.240
And and then I found out this is called algorithm aversion.
00:06:28.399 --> 00:06:38.240
Uh so the first paper to cover this, uh, what we would kind of consider the more seminal or or popular paper around this is a paper by um what do we have?
00:06:38.399 --> 00:06:43.279
Berkeley Diet Wurst, uh, Joseph Simmons, and Cade Massey.
00:06:43.439 --> 00:06:46.319
So I know Kay, uh recognize Cade Massey.
00:06:46.480 --> 00:06:50.720
They're all from uh uh Penn, so University of Pennsylvania.
00:06:50.959 --> 00:06:54.160
And um they wrote a paper called Algorithm Inversion.
00:06:54.480 --> 00:06:57.680
People erroneously avoid algorithms after seeing them.
00:06:59.360 --> 00:07:02.800
And so I go, that sounds like me.
00:07:03.040 --> 00:07:06.560
People erroneously avoid algorithms after seeing them error.
00:07:06.720 --> 00:07:08.720
Uh, this was a paper in 2015.
00:07:08.879 --> 00:07:17.439
This looks like Journal of Experimental Psychology, JEP again, what we call that in academic ranks.
00:07:17.600 --> 00:07:37.279
So a nice journal, and uh basically they showed within five different settings, they had an algorithm that would help predict future um MBA salary, student performance, future outcomes of graduates getting admitted into the school.
00:07:37.600 --> 00:07:41.680
And they compared that to people's judgment.
00:07:41.839 --> 00:07:56.399
And the way that they calibrated all five of their studies in this paper is they objectively had the algorithm as more accurate than any human prediction performance could give.
00:07:56.720 --> 00:08:12.639
And they also put in that error and they waited to see how quickly that response shifted away from algorithmic preferences to human preferences once they were betrayed by that error.
00:08:12.879 --> 00:08:19.839
And again, they found out that um one mistake, people immediately abandoned it.
00:08:20.079 --> 00:08:28.959
No second chances, no strike three, it's one and done, and it's a very aggressive move to the opposite way.
00:08:29.120 --> 00:08:32.480
And that's where we found this is algorithm aversion.
00:08:33.120 --> 00:08:36.320
And so we go, well, why is that happening?
00:08:36.480 --> 00:08:38.480
I've also had to think about that myself.
00:08:38.639 --> 00:08:44.639
But if you think you go back and go, you know, with humans, what do we expect from humans?
00:08:45.120 --> 00:08:48.399
And we're flawed, we're gonna make mistakes.
00:08:48.639 --> 00:08:54.960
When I think about ChatGPT, we think, well, they're beyond that, they're computers.
00:08:55.039 --> 00:09:00.000
And so my expectation level is they're gonna be perfect.
00:09:00.240 --> 00:09:06.960
And I always say I I love low expectations, I thrive in a low expectation environment.
00:09:07.120 --> 00:09:13.519
I hope you have low expectations for this podcast because then I can outperform and go, holy cow, I learned a lot.
00:09:13.679 --> 00:09:19.039
Um, once you get high expectations, like, oh, this guy's master expert, then you're gonna go, that's disappointing.
00:09:19.120 --> 00:09:20.559
He didn't really give me that much.
00:09:20.720 --> 00:09:26.960
Um, and so we think about these reference points here as well, and go, oh, well, that actually makes a big deal.
00:09:27.120 --> 00:09:28.000
It makes a difference.
00:09:28.159 --> 00:09:31.200
Because of human errors, you go, all right, well, it happens.
00:09:31.360 --> 00:09:37.600
Human nature, um, algorithm misses, that's below your threshold of perfection.
00:09:37.840 --> 00:09:39.120
That's negative reaction.
00:09:39.200 --> 00:09:41.360
We're gonna remember that very significantly.
00:09:41.600 --> 00:09:50.480
So it's uh a bit asymmetric here, whether information comes from a human or from the uh algorithm.
00:09:50.720 --> 00:09:52.720
So some theories that explain this.
00:09:52.879 --> 00:09:55.200
Number one, perfection expectations.
00:09:55.360 --> 00:10:01.919
This is uh also kind of tied into our reference points, what we consider gains and losses.
00:10:02.799 --> 00:10:05.279
That's called perfection expectations.
00:10:05.759 --> 00:10:06.879
Loss of control.
00:10:07.120 --> 00:10:15.200
There's another component here where we have this illusion of control where we want to be in control of our own decision making, our own destiny.
00:10:15.440 --> 00:10:31.039
This is why we do weird shit during our football games, because I think we're gonna control this by wearing the jersey or standing in a certain place while it's third and long because that's where they got first down on third and long before.
00:10:31.279 --> 00:10:49.919
Um, and so this loss of control when we're we're giving this uh away, we're we're losing our control, and and so that potentially, even if the computer is better, um, could explain why we dislike surrendering our decisions um fully to these algorithms.
00:10:50.960 --> 00:10:52.320
What about our egos?
00:10:52.720 --> 00:10:58.080
If pr the the computer is right and I'm not as right, that's a damage to egos.
00:10:58.399 --> 00:11:12.799
And the research has also found that more experts, so if you're an expert in your field and you're proven wrong or the computer outperforms you, then there's a damage to ego as well.
00:11:12.879 --> 00:11:16.000
So you've lost control, you've lost some ego.
00:11:16.399 --> 00:11:32.799
And this is where if you are in finance or if you're helping people um with their financial decisions, then all of a sudden you are in conflict now with your expertise and what AI might be telling the same clients and individuals.
00:11:33.440 --> 00:11:38.720
And this is where we're gonna have various various levels of humility around this.
00:11:38.879 --> 00:11:52.799
And there is a way to try to compliment them both, but anybody who's in the financial industry will tell you that their clients are most likely checking AI along with the responses that they are getting.
00:11:52.960 --> 00:12:02.559
Or maybe they're just blindly following AI, and the advisor has to somehow get them out and reroute them in the places that AI might be misleading them.
00:12:03.600 --> 00:12:10.559
Um, and then another one, this is the final one we'll talk about, is this uh betrayal aversion.
00:12:10.799 --> 00:12:12.399
And this is what I felt.
00:12:12.480 --> 00:12:14.399
I can't, I just have to explain it.
00:12:14.720 --> 00:12:18.320
People feel betrayed when technology disappoints them.
00:12:18.639 --> 00:12:27.360
And I don't have that same reaction when humans were to not give me the right information.
00:12:27.840 --> 00:12:30.960
Uh, and so we had some follow-up research on this.
00:12:31.279 --> 00:12:42.159
And uh, if we can actually have some sort of influence, I'm adjusting the algorithm, this can reduce our algorithm inversion.
00:12:42.320 --> 00:13:01.919
And this is where I found interesting with Chat GPT is okay, I've had this negative experience, I had a really strong and negative reaction to this, but now is there a way for me to go back in and teach this to the learning model where they don't get that wrong again?
00:13:02.159 --> 00:13:06.559
And so I use that as an opportunity to go back.
00:13:06.720 --> 00:13:24.000
I don't, I don't think I was very um tactful with how I told Chat GB team, but I told them it was total bullshit what they told me, and that next time, if they don't know the answer, tell me they don't know instead of giving me the lie.
00:13:24.480 --> 00:13:35.679
And that led to a real improvement not only of what the output was, it was also how I prompt, an improvement in how I prompt.
00:13:35.840 --> 00:13:44.240
And so garbage in, garbage out, if you give it better prompts, it'll give you way more insightful information that is going to be helpful.
00:13:44.399 --> 00:13:46.159
And so that's what I found.
00:13:46.240 --> 00:13:54.159
And this is what is helping reduce our algorithm aversion is being able to go in and make those adjustments.
00:13:54.240 --> 00:13:56.639
And that's what research has found as well.
00:13:56.879 --> 00:14:04.480
So in our world of finance, um, we now have robo advisors, um, AI is doing our portfolio management, our stock picking.
00:14:04.720 --> 00:14:06.320
So where do we fit in?
00:14:06.399 --> 00:14:08.320
Where do the humans fit into this?
00:14:08.799 --> 00:14:15.679
And I say our real value add is obviously from an emotional standpoint.
00:14:15.919 --> 00:14:26.799
Um, don't think AI is there when we're having that trouble of, hey, look, I'm ready to sell my entire portfolio.
00:14:27.200 --> 00:14:28.559
What do I do?
00:14:28.799 --> 00:14:36.399
What I'll have to check that with AI because now I like wonder what AI would tell me after all the different prompts it's told me to challenge me.
00:14:36.559 --> 00:14:43.200
You know, it always seems to revert back to confirmation bias of making me feel, hey, I think you're on the right.
00:14:43.519 --> 00:14:44.399
Well, that's a great idea.
00:14:44.559 --> 00:14:47.200
This could be one of your best episodes yet.
00:14:47.440 --> 00:14:52.559
Um I wonder how much it would push back if I say, hey, I'm gonna sell everything.
00:14:52.720 --> 00:15:05.039
Would it how far would it push me compared to a financial advisor if I were to call them on Monday and say, hey, I'm looking to go all cash, give me out, I'm ready to go.
00:15:05.200 --> 00:15:09.279
Uh so again, I think that's where the advantage lies with the humans.
00:15:09.440 --> 00:15:15.120
Uh at the same time, we can what else?
00:15:16.879 --> 00:15:17.600
Zillow.
00:15:18.799 --> 00:15:20.320
Zillow estimates.
00:15:20.960 --> 00:15:30.559
That's another one that makes you feel real good until you actually put it into the market and you realize that this estimate that Zillow gave you is not the same thing.
00:15:31.120 --> 00:15:32.799
Alright, so here's the thing.
00:15:33.120 --> 00:15:40.559
Automation bias, trust automation too much, algorithm inversion, we don't trust it at all.
00:15:40.879 --> 00:15:42.399
We don't trust it enough.
00:15:42.559 --> 00:15:44.080
Um, both are wrong.
00:15:44.559 --> 00:15:47.679
Now, where do we find ourselves in the middle here?
00:15:47.919 --> 00:15:51.840
And this is where we want to to to calibrate this.
00:15:52.159 --> 00:15:58.639
Now, the way I think about this is through framing, and I've had to take myself through this after my bad experience.
00:15:58.960 --> 00:16:03.759
Literally, it was like it was one time uh that it really, really betrayed me.
00:16:03.919 --> 00:16:05.600
But you can see it's still sticking here.
00:16:06.240 --> 00:16:16.080
Um we say, yeah, the machine will make mistakes, then we're gonna be averse to the algorithm.
00:16:16.639 --> 00:16:24.559
But at the end of the day, can we come back and say, look, which one is statistically going to be more reliable?
00:16:25.120 --> 00:16:32.960
And so the way we're setting this up, or where we're framing this question, is which one is gonna be more statistically reliable?
00:16:33.679 --> 00:16:39.039
And which one can I use, even though I can't have that expectation of perfection.
00:16:39.279 --> 00:16:57.200
So that gets me into this sweet spot of not 100% relying on it, but I've got to screen just enough where it's helping me, and I've got to reprompt my prompts to reconfirm.
00:16:57.360 --> 00:16:59.440
We talked about this uh on the last episode.
00:16:59.759 --> 00:17:12.880
Sometimes the AI prompts are like your teenaged boys, where you thought you told them once, and you realize that you have to tell them over and over again because they will quickly forget.
00:17:13.119 --> 00:17:16.160
And so you have to remind them about the prompts.
00:17:16.319 --> 00:17:19.359
Remember, you're gonna be challenging me, you're gonna be pushing back.
00:17:19.519 --> 00:17:23.599
I don't need to feel good all the time about what I'm trying to ask you.
00:17:23.920 --> 00:17:25.200
These kind of things.
00:17:25.440 --> 00:17:28.480
And um finding that sweet spot.
00:17:28.559 --> 00:17:30.480
So here's where we're gonna tie this all on.
00:17:34.960 --> 00:17:35.920
We already know.
00:17:36.079 --> 00:17:40.000
Judge algorithms by averages, not anecdotes.
00:17:40.559 --> 00:17:44.240
Um where humans still add value.
00:17:44.880 --> 00:17:48.799
This is where we talked about the emotional component.
00:17:49.279 --> 00:18:03.440
Um if you're not giving it good prompts, context a lot of that emotional decision making still here in 2026 favors itself in in favor of humans over AI.
00:18:03.599 --> 00:18:10.960
But again, AI's catching up, and it's all about how good or bad are are the prompts that you're giving it.
00:18:11.519 --> 00:18:15.519
Um and then calibrate your trust instead of making it binary.
00:18:15.680 --> 00:18:16.880
That's that's pretty good.
00:18:17.039 --> 00:18:22.799
And so as we wrap this up, the way I like to think about this is is within frameworks here.
00:18:22.960 --> 00:18:28.000
So we know we have the two posts, yes and no, total trust, no trust at all.
00:18:28.400 --> 00:18:30.240
Let's look at it as a continuum.
00:18:30.319 --> 00:18:33.119
We're gonna add a couple more pieces in the middle here.
00:18:33.279 --> 00:18:38.319
So blind rejection on one end, blind acceptance on the other.
00:18:39.119 --> 00:18:41.039
Way more than beat that one up.
00:18:41.119 --> 00:18:47.839
But in the middle, healthy skepticism or appropriate reliance.
00:18:48.480 --> 00:18:53.440
And you can see in between healthy skepticism is still closer to not trusting.
00:18:53.599 --> 00:18:57.200
Healthy reliance is over by the blind trust.
00:18:57.440 --> 00:19:13.839
That's where I would really, my ideal spot is in that area of that healthy reliance where you can still lean on it to really help you think better and to enhance your life, to make you more productive without putting your blind faith in it.
00:19:14.000 --> 00:19:17.200
So I had my um project that I was working on.
00:19:17.359 --> 00:19:31.039
I tell you all about the bad stuff, but I had a this project where I actually ran the analysis and was coming up with um Monte Carlo simulation on my own for this paper using 50,000 samples.
00:19:31.279 --> 00:19:50.559
And I had everything calibrated, all of the returns and deviations and correlations were all matching from different things, um, which are all green lights that you are set this up correctly and you have have done this right, except for one correlation.
00:19:50.720 --> 00:19:54.240
Oh god, why is this correlation off?
00:19:54.559 --> 00:19:55.279
So I go in.
00:19:55.440 --> 00:20:02.559
I was just like, I finally threw it into ChatGPT, and I was like, why is this correlation missing?
00:20:02.880 --> 00:20:13.920
And it found cell and one of the tabs, phase six, where I had put in the wrong like the the parentheses.
00:20:14.079 --> 00:20:22.880
So it didn't follow the mathematical um hierarchy, and it was coding it wrong.
00:20:22.960 --> 00:20:29.359
So I had one cell wrong, and it was all based on where I was putting my parentheses, and that was causing this error.
00:20:29.519 --> 00:20:31.119
And I was really kidding me.
00:20:31.359 --> 00:20:38.720
Found in within literally two seconds, I did it, re-ran it, everything all back in place, spot on.
00:20:38.880 --> 00:20:46.240
And so I'm going, okay, yes, you can use this really, really effectively.
00:20:46.480 --> 00:20:50.160
Um, but it has to be coupled with your own brain power.
00:20:50.480 --> 00:20:55.200
I think all conversations around AI eventually lead us to this.
00:20:55.519 --> 00:21:16.720
But at least I've really, in my own, for my own edification, I wanted to know what this actually is called, who's done the research on this before, and formalize the continue continuum of where do we fall ourselves on this space?
00:21:16.799 --> 00:21:18.480
And where do others fall?
00:21:18.640 --> 00:21:25.359
So then we can at least be able to have awareness on others where they fall.
00:21:25.519 --> 00:21:28.400
Oh, yeah, they are in blind rejection mode.
00:21:28.480 --> 00:21:37.599
And so maybe just trying to get them to a little skepticism uh around this, but still dipping a toe into using this machine.
00:21:37.920 --> 00:21:40.079
We can now use it in that way.
00:21:40.319 --> 00:21:45.279
So with that, I'm going to get off my perch here and move on with life.
00:21:45.440 --> 00:21:47.759
I'm going to forgive AI about its mistake.
00:21:47.920 --> 00:21:49.279
We'll continue to teach it.
00:21:49.359 --> 00:21:54.559
Um, but if you're liking what you hear, if you made it this far, thank you very much for doing so.
00:21:54.720 --> 00:22:00.559
Uh, we'll urge you to uh or encourage you to like, subscribe, comment.
00:22:00.720 --> 00:22:08.880
This gets the algorithm motivated to get more people involved in these hopefully really interesting conversations for you.
00:22:09.039 --> 00:22:18.000
Also, make sure right below in the show notes, you're gonna see a way to ask me a question or write a comment straight to me.
00:22:18.160 --> 00:22:19.920
You can leave a voicemail.
00:22:20.160 --> 00:22:22.079
It's real simple.
00:22:22.240 --> 00:22:28.480
And I'm starting to make my episodes around QA's, which have become very, very interesting to me.
00:22:28.720 --> 00:22:32.799
Newsletter also there on the moneystrong.net.
00:22:32.880 --> 00:22:46.480
You can go right there, get involved in a newsletter, stay up to date on what we're talking about, get it in written form, and don't miss anything as we're rolling out new behavioral finance concepts, topics every single week.
00:22:46.640 --> 00:22:47.519
So thanks again.
00:22:47.680 --> 00:22:48.960
Hope you have a wonderful day.
00:22:49.119 --> 00:22:50.480
We'll see you on the next episode.
00:22:50.559 --> 00:22:51.440
Take care, everyone.
00:22:51.519 --> 00:22:52.559
Bye-bye.
00:22:54.960 --> 00:23:00.240
Thank you for tuning in to another episode of the Brian Fultis Behavioral Finance Podcast.
00:23:00.400 --> 00:23:07.519
We hope you found our exploration into the fascinating world of human behavior and finance, both enlightening and thought provoking.
00:23:07.759 --> 00:23:10.079
Be sure to subscribe for future episodes.
00:23:10.240 --> 00:23:14.160
And until next time, stay curious and financially savvy.
00:00:04.080 --> 00:00:15.279
Welcome to the Brian Fultis Behavioral Finance Podcast, where we unravel the mysteries of behavioral finance and unlock the secrets to making smarter, more informed decisions with your money.
00:00:15.519 --> 00:00:17.359
Now, here's your host, Dr.
00:00:17.440 --> 00:00:18.719
Brian Fultis.
00:00:24.879 --> 00:00:26.320
Welcome back, everybody.
00:00:26.480 --> 00:00:33.600
We're going to continue our conversation on using technology and how much should we trust it?
00:00:33.920 --> 00:00:36.960
Or should we never trust it at all?
00:00:37.280 --> 00:00:46.799
And that's where we want to wind up here having a good answer and having an actual framework on where we fit on this continuum by the end of this episode.
00:00:47.039 --> 00:01:15.040
And this is part two of the series on automation bias, which is putting too much trust in technology, to today's topic of algorithm inversion, which means when an algorithm gives you bad information, you have a very strong negative reaction to that, much stronger than if a human were to give you bad information.
00:01:15.359 --> 00:01:18.400
And so that's what we're going to be talking about today.
00:01:18.640 --> 00:01:28.799
So remember, as we said, automation bias, we were looking at the proceedings and this summary of automation bias using pilots back in the 90s.
00:01:29.040 --> 00:01:48.239
And we found that the two types of errors from the automation bias, omission, so not getting information from the cockpit or the computer, and also commission, where it gives you the wrong bit of information and you file or follow that blindly without checking your own information.
00:01:48.400 --> 00:01:52.959
Those are the two types of errors that lead to this automation bias.
00:01:53.760 --> 00:01:58.640
Same authors that I referred to last week, Kathleen Mosier and Linda Skitka.
00:01:58.799 --> 00:02:08.960
They both uh additionally, I made some extra notes that if you add a second person, that that does not necessarily help you alleviate it.
00:02:09.039 --> 00:02:23.199
It just basically gives you another excuse not to be vigilant and to just trust what the computer tells you, even though you would think another set of eyes would reduce that quite significantly.
00:02:23.520 --> 00:02:37.120
They also found that uh teaching about this automation bias actually does reduce people's bias and they actually will question and be more vigilant.
00:02:37.280 --> 00:02:56.479
So it's actually very encouraging for us because as we are learning about this, we're also learning that with awareness comes that ability to say that, hey, look, I'm using my type one thinking, my automatic way of thinking, maybe a little bit too much.
00:02:56.639 --> 00:03:02.719
Maybe it's time for me to really critically think or me be more critical in this particular area.
00:03:02.879 --> 00:03:17.120
And it's through that awareness that we can actually get to that spot where we can step in, be vigilant, or double check that the information that we are receiving from technology is actually correct.
00:03:17.919 --> 00:03:25.039
And so when we're talking about the messaging here, we have to realize that it is not about AI.
00:03:25.599 --> 00:03:28.400
Um it's a lot about our ego.
00:03:28.719 --> 00:03:36.159
And so we are way more willing to forgive other humans than we are machines.
00:03:36.960 --> 00:03:47.120
And I had this happen in my research work this summer, where I was writing, I'm writing this paper on uh withdrawal rate rules.
00:03:47.280 --> 00:03:53.280
And you're gonna hear a whole series on what I found this summer because I'm just on the back end of it.
00:03:53.439 --> 00:04:07.680
But I remember one of the days I was actually hadn't read this paper for a little while and just wanted to go back in and was asking ChatGPT to give me that summary of their methodology and what they were doing, because I just needed a refresher.
00:04:07.759 --> 00:04:12.240
I didn't want to go and find the paper and dig in and go through it all myself.
00:04:12.319 --> 00:04:14.159
So I was kind of leaning on them.
00:04:14.479 --> 00:04:34.800
And ChatGPT came back with a very complete, non-complete answer saying that this study actually went and did a a Monte Carlo Carlo simulation and that it was different from all the other papers.
00:04:35.360 --> 00:04:42.399
And it's through this the findings and then just knocked out like these bullet points and was fairly believable.
00:04:42.480 --> 00:04:54.560
You go, okay, well, that makes sense that um it may that somebody went to a Monte Carlo simulation with this data and that they would use it to basically recheck these findings.
00:04:54.879 --> 00:04:59.279
Until I had to step in and go, I don't think that's true.
00:04:59.439 --> 00:05:04.480
That sounds way off from the paper that I read a while back.
00:05:04.879 --> 00:05:16.319
Well, sure enough, I look at that paper and go, that's not even close to what they use previous empirical data, like like historical returns in their analysis.
00:05:16.639 --> 00:05:23.839
And not even one single mention, let alone their main finding around this Monte Carlo simulation.
00:05:24.000 --> 00:05:44.879
And I go, I felt so betrayed by AI, and I was really really angry about it, going, my reaction was way stronger than somebody saying, Hey, look, I think it's this, or or just basically lying, or whatever.
00:05:45.040 --> 00:05:48.800
If a human were to tell me, oh wait, I had that confused with something else.
00:05:48.879 --> 00:05:49.680
I'm sorry.
00:05:54.319 --> 00:06:05.759
You, but the way that AI presented that to me and and just gave it as this gospel, and I was like, I felt really betrayed by this.
00:06:05.920 --> 00:06:08.079
So what we're talking about here, that's what started.
00:06:08.160 --> 00:06:16.480
I go, man, I really had a strong negative response to this, saying, I'm never gonna have ChatGPT help me with any of these summaries.
00:06:16.560 --> 00:06:18.959
I gotta do it all myself, just like the old days.
00:06:19.120 --> 00:06:23.839
And it's like, okay, are you are you overreacting maybe a little bit?
00:06:24.160 --> 00:06:28.240
And and then I found out this is called algorithm aversion.
00:06:28.399 --> 00:06:38.240
Uh so the first paper to cover this, uh, what we would kind of consider the more seminal or or popular paper around this is a paper by um what do we have?
00:06:38.399 --> 00:06:43.279
Berkeley Diet Wurst, uh, Joseph Simmons, and Cade Massey.
00:06:43.439 --> 00:06:46.319
So I know Kay, uh recognize Cade Massey.
00:06:46.480 --> 00:06:50.720
They're all from uh uh Penn, so University of Pennsylvania.
00:06:50.959 --> 00:06:54.160
And um they wrote a paper called Algorithm Inversion.
00:06:54.480 --> 00:06:57.680
People erroneously avoid algorithms after seeing them.
00:06:59.360 --> 00:07:02.800
And so I go, that sounds like me.
00:07:03.040 --> 00:07:06.560
People erroneously avoid algorithms after seeing them error.
00:07:06.720 --> 00:07:08.720
Uh, this was a paper in 2015.
00:07:08.879 --> 00:07:17.439
This looks like Journal of Experimental Psychology, JEP again, what we call that in academic ranks.
00:07:17.600 --> 00:07:37.279
So a nice journal, and uh basically they showed within five different settings, they had an algorithm that would help predict future um MBA salary, student performance, future outcomes of graduates getting admitted into the school.
00:07:37.600 --> 00:07:41.680
And they compared that to people's judgment.
00:07:41.839 --> 00:07:56.399
And the way that they calibrated all five of their studies in this paper is they objectively had the algorithm as more accurate than any human prediction performance could give.
00:07:56.720 --> 00:08:12.639
And they also put in that error and they waited to see how quickly that response shifted away from algorithmic preferences to human preferences once they were betrayed by that error.
00:08:12.879 --> 00:08:19.839
And again, they found out that um one mistake, people immediately abandoned it.
00:08:20.079 --> 00:08:28.959
No second chances, no strike three, it's one and done, and it's a very aggressive move to the opposite way.
00:08:29.120 --> 00:08:32.480
And that's where we found this is algorithm aversion.
00:08:33.120 --> 00:08:36.320
And so we go, well, why is that happening?
00:08:36.480 --> 00:08:38.480
I've also had to think about that myself.
00:08:38.639 --> 00:08:44.639
But if you think you go back and go, you know, with humans, what do we expect from humans?
00:08:45.120 --> 00:08:48.399
And we're flawed, we're gonna make mistakes.
00:08:48.639 --> 00:08:54.960
When I think about ChatGPT, we think, well, they're beyond that, they're computers.
00:08:55.039 --> 00:09:00.000
And so my expectation level is they're gonna be perfect.
00:09:00.240 --> 00:09:06.960
And I always say I I love low expectations, I thrive in a low expectation environment.
00:09:07.120 --> 00:09:13.519
I hope you have low expectations for this podcast because then I can outperform and go, holy cow, I learned a lot.
00:09:13.679 --> 00:09:19.039
Um, once you get high expectations, like, oh, this guy's master expert, then you're gonna go, that's disappointing.
00:09:19.120 --> 00:09:20.559
He didn't really give me that much.
00:09:20.720 --> 00:09:26.960
Um, and so we think about these reference points here as well, and go, oh, well, that actually makes a big deal.
00:09:27.120 --> 00:09:28.000
It makes a difference.
00:09:28.159 --> 00:09:31.200
Because of human errors, you go, all right, well, it happens.
00:09:31.360 --> 00:09:37.600
Human nature, um, algorithm misses, that's below your threshold of perfection.
00:09:37.840 --> 00:09:39.120
That's negative reaction.
00:09:39.200 --> 00:09:41.360
We're gonna remember that very significantly.
00:09:41.600 --> 00:09:50.480
So it's uh a bit asymmetric here, whether information comes from a human or from the uh algorithm.
00:09:50.720 --> 00:09:52.720
So some theories that explain this.
00:09:52.879 --> 00:09:55.200
Number one, perfection expectations.
00:09:55.360 --> 00:10:01.919
This is uh also kind of tied into our reference points, what we consider gains and losses.
00:10:02.799 --> 00:10:05.279
That's called perfection expectations.
00:10:05.759 --> 00:10:06.879
Loss of control.
00:10:07.120 --> 00:10:15.200
There's another component here where we have this illusion of control where we want to be in control of our own decision making, our own destiny.
00:10:15.440 --> 00:10:31.039
This is why we do weird shit during our football games, because I think we're gonna control this by wearing the jersey or standing in a certain place while it's third and long because that's where they got first down on third and long before.
00:10:31.279 --> 00:10:49.919
Um, and so this loss of control when we're we're giving this uh away, we're we're losing our control, and and so that potentially, even if the computer is better, um, could explain why we dislike surrendering our decisions um fully to these algorithms.
00:10:50.960 --> 00:10:52.320
What about our egos?
00:10:52.720 --> 00:10:58.080
If pr the the computer is right and I'm not as right, that's a damage to egos.
00:10:58.399 --> 00:11:12.799
And the research has also found that more experts, so if you're an expert in your field and you're proven wrong or the computer outperforms you, then there's a damage to ego as well.
00:11:12.879 --> 00:11:16.000
So you've lost control, you've lost some ego.
00:11:16.399 --> 00:11:32.799
And this is where if you are in finance or if you're helping people um with their financial decisions, then all of a sudden you are in conflict now with your expertise and what AI might be telling the same clients and individuals.
00:11:33.440 --> 00:11:38.720
And this is where we're gonna have various various levels of humility around this.
00:11:38.879 --> 00:11:52.799
And there is a way to try to compliment them both, but anybody who's in the financial industry will tell you that their clients are most likely checking AI along with the responses that they are getting.
00:11:52.960 --> 00:12:02.559
Or maybe they're just blindly following AI, and the advisor has to somehow get them out and reroute them in the places that AI might be misleading them.
00:12:03.600 --> 00:12:10.559
Um, and then another one, this is the final one we'll talk about, is this uh betrayal aversion.
00:12:10.799 --> 00:12:12.399
And this is what I felt.
00:12:12.480 --> 00:12:14.399
I can't, I just have to explain it.
00:12:14.720 --> 00:12:18.320
People feel betrayed when technology disappoints them.
00:12:18.639 --> 00:12:27.360
And I don't have that same reaction when humans were to not give me the right information.
00:12:27.840 --> 00:12:30.960
Uh, and so we had some follow-up research on this.
00:12:31.279 --> 00:12:42.159
And uh, if we can actually have some sort of influence, I'm adjusting the algorithm, this can reduce our algorithm inversion.
00:12:42.320 --> 00:13:01.919
And this is where I found interesting with Chat GPT is okay, I've had this negative experience, I had a really strong and negative reaction to this, but now is there a way for me to go back in and teach this to the learning model where they don't get that wrong again?
00:13:02.159 --> 00:13:06.559
And so I use that as an opportunity to go back.
00:13:06.720 --> 00:13:24.000
I don't, I don't think I was very um tactful with how I told Chat GB team, but I told them it was total bullshit what they told me, and that next time, if they don't know the answer, tell me they don't know instead of giving me the lie.
00:13:24.480 --> 00:13:35.679
And that led to a real improvement not only of what the output was, it was also how I prompt, an improvement in how I prompt.
00:13:35.840 --> 00:13:44.240
And so garbage in, garbage out, if you give it better prompts, it'll give you way more insightful information that is going to be helpful.
00:13:44.399 --> 00:13:46.159
And so that's what I found.
00:13:46.240 --> 00:13:54.159
And this is what is helping reduce our algorithm aversion is being able to go in and make those adjustments.
00:13:54.240 --> 00:13:56.639
And that's what research has found as well.
00:13:56.879 --> 00:14:04.480
So in our world of finance, um, we now have robo advisors, um, AI is doing our portfolio management, our stock picking.
00:14:04.720 --> 00:14:06.320
So where do we fit in?
00:14:06.399 --> 00:14:08.320
Where do the humans fit into this?
00:14:08.799 --> 00:14:15.679
And I say our real value add is obviously from an emotional standpoint.
00:14:15.919 --> 00:14:26.799
Um, don't think AI is there when we're having that trouble of, hey, look, I'm ready to sell my entire portfolio.
00:14:27.200 --> 00:14:28.559
What do I do?
00:14:28.799 --> 00:14:36.399
What I'll have to check that with AI because now I like wonder what AI would tell me after all the different prompts it's told me to challenge me.
00:14:36.559 --> 00:14:43.200
You know, it always seems to revert back to confirmation bias of making me feel, hey, I think you're on the right.
00:14:43.519 --> 00:14:44.399
Well, that's a great idea.
00:14:44.559 --> 00:14:47.200
This could be one of your best episodes yet.
00:14:47.440 --> 00:14:52.559
Um I wonder how much it would push back if I say, hey, I'm gonna sell everything.
00:14:52.720 --> 00:15:05.039
Would it how far would it push me compared to a financial advisor if I were to call them on Monday and say, hey, I'm looking to go all cash, give me out, I'm ready to go.
00:15:05.200 --> 00:15:09.279
Uh so again, I think that's where the advantage lies with the humans.
00:15:09.440 --> 00:15:15.120
Uh at the same time, we can what else?
00:15:16.879 --> 00:15:17.600
Zillow.
00:15:18.799 --> 00:15:20.320
Zillow estimates.
00:15:20.960 --> 00:15:30.559
That's another one that makes you feel real good until you actually put it into the market and you realize that this estimate that Zillow gave you is not the same thing.
00:15:31.120 --> 00:15:32.799
Alright, so here's the thing.
00:15:33.120 --> 00:15:40.559
Automation bias, trust automation too much, algorithm inversion, we don't trust it at all.
00:15:40.879 --> 00:15:42.399
We don't trust it enough.
00:15:42.559 --> 00:15:44.080
Um, both are wrong.
00:15:44.559 --> 00:15:47.679
Now, where do we find ourselves in the middle here?
00:15:47.919 --> 00:15:51.840
And this is where we want to to to calibrate this.
00:15:52.159 --> 00:15:58.639
Now, the way I think about this is through framing, and I've had to take myself through this after my bad experience.
00:15:58.960 --> 00:16:03.759
Literally, it was like it was one time uh that it really, really betrayed me.
00:16:03.919 --> 00:16:05.600
But you can see it's still sticking here.
00:16:06.240 --> 00:16:16.080
Um we say, yeah, the machine will make mistakes, then we're gonna be averse to the algorithm.
00:16:16.639 --> 00:16:24.559
But at the end of the day, can we come back and say, look, which one is statistically going to be more reliable?
00:16:25.120 --> 00:16:32.960
And so the way we're setting this up, or where we're framing this question, is which one is gonna be more statistically reliable?
00:16:33.679 --> 00:16:39.039
And which one can I use, even though I can't have that expectation of perfection.
00:16:39.279 --> 00:16:57.200
So that gets me into this sweet spot of not 100% relying on it, but I've got to screen just enough where it's helping me, and I've got to reprompt my prompts to reconfirm.
00:16:57.360 --> 00:16:59.440
We talked about this uh on the last episode.
00:16:59.759 --> 00:17:12.880
Sometimes the AI prompts are like your teenaged boys, where you thought you told them once, and you realize that you have to tell them over and over again because they will quickly forget.
00:17:13.119 --> 00:17:16.160
And so you have to remind them about the prompts.
00:17:16.319 --> 00:17:19.359
Remember, you're gonna be challenging me, you're gonna be pushing back.
00:17:19.519 --> 00:17:23.599
I don't need to feel good all the time about what I'm trying to ask you.
00:17:23.920 --> 00:17:25.200
These kind of things.
00:17:25.440 --> 00:17:28.480
And um finding that sweet spot.
00:17:28.559 --> 00:17:30.480
So here's where we're gonna tie this all on.
00:17:34.960 --> 00:17:35.920
We already know.
00:17:36.079 --> 00:17:40.000
Judge algorithms by averages, not anecdotes.
00:17:40.559 --> 00:17:44.240
Um where humans still add value.
00:17:44.880 --> 00:17:48.799
This is where we talked about the emotional component.
00:17:49.279 --> 00:18:03.440
Um if you're not giving it good prompts, context a lot of that emotional decision making still here in 2026 favors itself in in favor of humans over AI.
00:18:03.599 --> 00:18:10.960
But again, AI's catching up, and it's all about how good or bad are are the prompts that you're giving it.
00:18:11.519 --> 00:18:15.519
Um and then calibrate your trust instead of making it binary.
00:18:15.680 --> 00:18:16.880
That's that's pretty good.
00:18:17.039 --> 00:18:22.799
And so as we wrap this up, the way I like to think about this is is within frameworks here.
00:18:22.960 --> 00:18:28.000
So we know we have the two posts, yes and no, total trust, no trust at all.
00:18:28.400 --> 00:18:30.240
Let's look at it as a continuum.
00:18:30.319 --> 00:18:33.119
We're gonna add a couple more pieces in the middle here.
00:18:33.279 --> 00:18:38.319
So blind rejection on one end, blind acceptance on the other.
00:18:39.119 --> 00:18:41.039
Way more than beat that one up.
00:18:41.119 --> 00:18:47.839
But in the middle, healthy skepticism or appropriate reliance.
00:18:48.480 --> 00:18:53.440
And you can see in between healthy skepticism is still closer to not trusting.
00:18:53.599 --> 00:18:57.200
Healthy reliance is over by the blind trust.
00:18:57.440 --> 00:19:13.839
That's where I would really, my ideal spot is in that area of that healthy reliance where you can still lean on it to really help you think better and to enhance your life, to make you more productive without putting your blind faith in it.
00:19:14.000 --> 00:19:17.200
So I had my um project that I was working on.
00:19:17.359 --> 00:19:31.039
I tell you all about the bad stuff, but I had a this project where I actually ran the analysis and was coming up with um Monte Carlo simulation on my own for this paper using 50,000 samples.
00:19:31.279 --> 00:19:50.559
And I had everything calibrated, all of the returns and deviations and correlations were all matching from different things, um, which are all green lights that you are set this up correctly and you have have done this right, except for one correlation.
00:19:50.720 --> 00:19:54.240
Oh god, why is this correlation off?
00:19:54.559 --> 00:19:55.279
So I go in.
00:19:55.440 --> 00:20:02.559
I was just like, I finally threw it into ChatGPT, and I was like, why is this correlation missing?
00:20:02.880 --> 00:20:13.920
And it found cell and one of the tabs, phase six, where I had put in the wrong like the the parentheses.
00:20:14.079 --> 00:20:22.880
So it didn't follow the mathematical um hierarchy, and it was coding it wrong.
00:20:22.960 --> 00:20:29.359
So I had one cell wrong, and it was all based on where I was putting my parentheses, and that was causing this error.
00:20:29.519 --> 00:20:31.119
And I was really kidding me.
00:20:31.359 --> 00:20:38.720
Found in within literally two seconds, I did it, re-ran it, everything all back in place, spot on.
00:20:38.880 --> 00:20:46.240
And so I'm going, okay, yes, you can use this really, really effectively.
00:20:46.480 --> 00:20:50.160
Um, but it has to be coupled with your own brain power.
00:20:50.480 --> 00:20:55.200
I think all conversations around AI eventually lead us to this.
00:20:55.519 --> 00:21:16.720
But at least I've really, in my own, for my own edification, I wanted to know what this actually is called, who's done the research on this before, and formalize the continue continuum of where do we fall ourselves on this space?
00:21:16.799 --> 00:21:18.480
And where do others fall?
00:21:18.640 --> 00:21:25.359
So then we can at least be able to have awareness on others where they fall.
00:21:25.519 --> 00:21:28.400
Oh, yeah, they are in blind rejection mode.
00:21:28.480 --> 00:21:37.599
And so maybe just trying to get them to a little skepticism uh around this, but still dipping a toe into using this machine.
00:21:37.920 --> 00:21:40.079
We can now use it in that way.
00:21:40.319 --> 00:21:45.279
So with that, I'm going to get off my perch here and move on with life.
00:21:45.440 --> 00:21:47.759
I'm going to forgive AI about its mistake.
00:21:47.920 --> 00:21:49.279
We'll continue to teach it.
00:21:49.359 --> 00:21:54.559
Um, but if you're liking what you hear, if you made it this far, thank you very much for doing so.
00:21:54.720 --> 00:22:00.559
Uh, we'll urge you to uh or encourage you to like, subscribe, comment.
00:22:00.720 --> 00:22:08.880
This gets the algorithm motivated to get more people involved in these hopefully really interesting conversations for you.
00:22:09.039 --> 00:22:18.000
Also, make sure right below in the show notes, you're gonna see a way to ask me a question or write a comment straight to me.
00:22:18.160 --> 00:22:19.920
You can leave a voicemail.
00:22:20.160 --> 00:22:22.079
It's real simple.
00:22:22.240 --> 00:22:28.480
And I'm starting to make my episodes around QA's, which have become very, very interesting to me.
00:22:28.720 --> 00:22:32.799
Newsletter also there on the moneystrong.net.
00:22:32.880 --> 00:22:46.480
You can go right there, get involved in a newsletter, stay up to date on what we're talking about, get it in written form, and don't miss anything as we're rolling out new behavioral finance concepts, topics every single week.
00:22:46.640 --> 00:22:47.519
So thanks again.
00:22:47.680 --> 00:22:48.960
Hope you have a wonderful day.
00:22:49.119 --> 00:22:50.480
We'll see you on the next episode.
00:22:50.559 --> 00:22:51.440
Take care, everyone.
00:22:51.519 --> 00:22:52.559
Bye-bye.
00:22:54.960 --> 00:23:00.240
Thank you for tuning in to another episode of the Brian Fultis Behavioral Finance Podcast.
00:23:00.400 --> 00:23:07.519
We hope you found our exploration into the fascinating world of human behavior and finance, both enlightening and thought provoking.
00:23:07.759 --> 00:23:10.079
Be sure to subscribe for future episodes.
00:23:10.240 --> 00:23:14.160
And until next time, stay curious and financially savvy.
