Cognitive Surrender: How AI Weaponizes Human Psychology

WEBVTT
00:00:00.239 --> 00:00:02.959
You're passing your thinking on to these AI tools.
00:00:02.959 --> 00:00:11.439
And I really like that term cognitive surrender because it it is us giving up a lot of our thinking and and kind of what made us human.
00:00:15.599 --> 00:00:18.079
Welcome to the audit, presented by IT Audit Labs.
00:00:18.079 --> 00:00:19.359
My name is Joshua Schmidt.
00:00:19.359 --> 00:00:22.239
I'm your co-host and producer, and we're joined by the usual suspects.
00:00:22.239 --> 00:00:28.800
Eric Brown, managing director and security engineer and director, Nick Mellum, coming from Texas.
00:00:28.800 --> 00:00:33.119
And today our guest is James McDowell, and he's coming from Tennessee, yeah?
00:00:33.119 --> 00:00:34.640
Yeah, absolutely.
00:00:34.640 --> 00:00:37.280
We were talking before this, doing some pre-production.
00:00:37.280 --> 00:00:43.359
Our conversation started to gravitate towards AI, weaponizing human psychology in unprecedented ways.
00:00:43.359 --> 00:00:55.920
And uh you had mentioned uh a few things that were really interesting to us in that regard because we've been focusing a lot on AI at IT audit labs, integrating that into a lot of our workflows and in even some upcoming projects.
00:00:55.920 --> 00:01:00.880
So, yeah, maybe you could give us a little background on yourself and um what's top of mind for you today.
00:01:01.200 --> 00:01:22.719
I think really importantly, like uh, you know, I bring an attitude to cybercrime investigations that really we're integrating psychology into our investigations and not only from the victim-centered response angle on how we should be treating victims or or survivors of cybercrime uh with more compassion than I think is typically treated.
00:01:22.719 --> 00:01:30.719
But then also a different lens in how we can think about offenders in in psychology.
00:01:30.719 --> 00:01:35.200
And so uh been doing that for about 10 years now, uh really enjoyed it.
00:01:35.200 --> 00:01:39.840
A lot of different experiences, as you all know in cyber, it it changes every day.
00:01:39.840 --> 00:01:49.200
Um and then yeah, adjunct professor, so working with the the next generation, and and it's always fun to see them uh delightful moment when when it clicks for them as well.
00:01:49.519 --> 00:01:57.200
James, I I spent time in the in the military and uh one of my best friends coming out of the military, he went a psychology path.
00:01:57.200 --> 00:02:01.280
And he now is a psychologist for the BA, amongst many other things.
00:02:01.280 --> 00:02:09.840
I'd have a lot of deep conversation, well relatively deep conversations about, you know, veterans, um, but then also, you know, people in the work, burnout, et cetera.
00:02:09.840 --> 00:02:15.120
Is there any trends that you're seeing now that are that are maybe new within the last five to ten years?
00:02:15.120 --> 00:02:21.280
You know, we got out of a really long, extensive war in the Middle East, and now we're shifting back in.
00:02:21.280 --> 00:02:25.120
If you've been paying attention to the news over the past couple days, cybercrime is up, right?
00:02:25.120 --> 00:02:26.400
All these things are happening.
00:02:26.400 --> 00:02:34.240
If you tie all that together, is there any trends from your perspective, maybe from your teaching um past and present?
00:02:34.639 --> 00:02:35.439
Yeah, absolutely.
00:02:35.439 --> 00:02:43.919
I think a really interesting trend, and and it's not my term, but cognitive surrender is what we've seen in the last five years with the growth of the AI tools.
00:02:43.919 --> 00:02:46.879
You're passing your thinking on to these AI tools.
00:02:46.879 --> 00:02:58.879
And and I really like that term cognitive surrender because it it is us giving up a lot of our thinking and and kind of what made us human up until a lot of these AI tools got really popular a few years ago.
00:02:58.879 --> 00:03:02.560
And I think that's a really interesting trend that we're only going to see more of.
00:03:02.560 --> 00:03:09.439
And then as you think about that in the context of cybercrime, like are we seeing a bigger uptick related to that?
00:03:09.439 --> 00:03:19.840
And I think there is correlation there, maybe not causation yet, but I uh not only are we seeing the scale explode with AI tools, but how we respond to the scale of those instances.
00:03:20.319 --> 00:03:31.120
So, James, along those lines, and I I know I was perusing your um your book a little earlier, your book's uh Forensic Psychology and the Human Side of Cybercrime.
00:03:31.120 --> 00:03:36.000
You talk about a framework in there called the wave theory.
00:03:36.000 --> 00:03:46.800
And could that wave theory um be a framework to look at this at scale from a psychological perspective?
00:03:46.800 --> 00:03:49.599
Or how would how would you relate the two?
00:03:49.919 --> 00:03:50.800
Yeah, absolutely.
00:03:50.800 --> 00:03:54.080
You're you're right on the money and appreciate the plug there for the book.
00:03:54.080 --> 00:04:08.479
Um, but yes, wave theory is the first attempt at unifying um how we think about cybercrime from the human aspect and really thinking through how offenders are exploiting our psychology and then how we're responding to it.
00:04:08.479 --> 00:04:09.680
Uh, so both sides.
00:04:09.680 --> 00:04:18.240
And and yeah, really weaponized effective vulnerability engineering uh wave theory is is that term and really the first step.
00:04:18.240 --> 00:04:28.480
And so I think there's going to be a lot of research coming out in the next decade around the psychology of that, and we'll continue to see cyber psychology grow as a field across the globe.
00:04:28.480 --> 00:04:30.879
And I'm really excited to be a part of that field.
00:04:30.879 --> 00:04:49.839
Um, and yet to your other question on how it relates to AI and what we're talking about, I think we're seeing how our brains evolved for the physical world and how we can ascertain, decide who we should trust in the physical world for the last thousands of years.
00:04:49.839 --> 00:04:57.439
And now all of a sudden, all of our the majority of our interactions are moved to the virtual digital world, and our brains did not evolve for that.
00:04:57.439 --> 00:05:09.839
And I think that's part of why we're seeing a lot of successful cybercrime exploits and a lot of survivors dealing with the fallout of that because we just as humans did not evolve for for what we're dealing with now.
00:05:10.800 --> 00:05:26.399
Is your your your book I I I got it um as an ebook through um it seemed like you had to go through a college kind of um textbook store, uh uh vital sources where I picked it up.
00:05:26.399 --> 00:05:34.079
I think there's a Kindle version on Amazon, but is is it mainly used as a college textbook?
00:05:34.720 --> 00:05:35.759
Yes, absolutely.
00:05:35.759 --> 00:05:42.879
And that's uh I think the if folks weren't anticipating that, it it's definitely more academic and definitely dense.
00:05:42.879 --> 00:05:44.800
Um I'll be the first to admit that.
00:05:44.800 --> 00:05:54.959
And I think that um really it's looking at the theory and proving up the theory based on research, and it's it's geared towards that college course, uh teaching it in a college classroom.
00:05:55.279 --> 00:06:13.199
Yeah, we we come from the the cyber side and a lot of things are are black and white of you know, we need to put these protections in place, we need to trim off these um permissions or what have you, and then you know, the the user in most cases is uh is a byproduct of that.
00:06:13.839 --> 00:06:15.600
That's exactly where I was going with this.
00:06:15.600 --> 00:06:24.399
Uh when I talk with James, you mentioned uh James, you mentioned uh a specific instance in a story with uh quite a substantial amount of money.
00:06:24.399 --> 00:06:30.240
I think it was out of Hong Kong where there was a deep fake, leveraging deep fakes to uh get some money out of some folks.
00:06:30.240 --> 00:06:34.560
Could could you share that story with us and then maybe we could go into some other examples, yeah.
00:06:35.040 --> 00:06:35.839
Yeah, absolutely.
00:06:35.839 --> 00:06:49.279
And and I think that's a really interested one and interested to hear everyone's perspective on it as we talk through it uh because it's one of the first instances from a few years ago where, like you mentioned, there was a company based out of Hong Kong, but really a global operation.
00:06:49.279 --> 00:06:56.160
And an individual that reported to the CFO uh got an interesting email on a Friday afternoon.
00:06:56.160 --> 00:07:00.240
It was uh requesting that money be transferred for an upcoming acquisition.
00:07:00.240 --> 00:07:05.439
All the context felt right, but the the way that it was phrased to the individual felt off.
00:07:05.439 --> 00:07:14.000
And, you know, $25 million, $200 Hong Kong dollars being asked for on a Friday afternoon raised some red flags in this individual's head.
00:07:14.000 --> 00:07:20.959
So they emailed the individual back, the the CFO of the company back and said, Hey, can we hop on a call and talk this through?
00:07:20.959 --> 00:07:24.160
They quickly got a link to a virtual meeting.
00:07:24.160 --> 00:07:31.120
They hopped on a call with not only the CFO, but three or four other executives from the company based around the world.
00:07:31.120 --> 00:07:37.519
They talked about the acquisition, talked about the money that they needed to wire to the CFO for the acquisition.
00:07:37.519 --> 00:07:41.759
They felt comfortable after that virtual call with multiple people from the company.
00:07:41.759 --> 00:07:49.519
So then they wired the money, got a logged off, went about their weekend, got a call Monday morning that said, What did you do?
00:07:49.519 --> 00:07:50.720
Where is all this money?
00:07:50.720 --> 00:07:58.639
Uh, come to find out, not only was the CFO email a phishing email, but all of the people on that call were deep fakes.
00:07:58.639 --> 00:08:11.519
And really, so that shows you the psychology that we're dealing with that uh you had a little bit of that uh social cognition where you had multiple people convincing you this was a good idea, or you know, multiple people convincing you this was a good idea.
00:08:11.519 --> 00:08:17.199
And I think that's a really interesting way and in which we see the adversary exploiting our psychology too.
00:08:17.519 --> 00:08:25.920
So not just that first layer of hesitancy where you go, oh, something feels off here, but they have backstops built in for that to to account for that.
00:08:25.920 --> 00:08:32.639
So you have to go several layers deep to discover that there's actually a threat actor behind something like this.
00:08:33.039 --> 00:08:33.840
Yeah, absolutely.
00:08:33.840 --> 00:08:50.320
And and as you all know, we've we'd spent years telling people don't trust the email, but you can trust uh pick up the phone, call people, and and you know, the the adversary was listening to us say that and came up with their scams to circumvent the the red flag approach that we uh we've been using for decades.
00:08:50.320 --> 00:08:52.960
Did you guys discuss what year this took place?
00:08:52.960 --> 00:08:54.639
What when was this?
00:08:54.639 --> 00:08:56.720
This was 2024.
00:08:56.720 --> 00:08:58.799
Okay, so it was relatively recent.
00:08:58.799 --> 00:09:09.840
Yeah, and it and it was right as deepfakes were were, you know, um the growth curve has been exponential, so they were still, I guess, bad compared to today's deepfakes, but good compared to what we had seen prior to that.
00:09:10.159 --> 00:09:18.879
So do you do you have a sense of whether this was one person operating multiple deepfakes or a whole team of folks spinning up different deepfakes for this one scenario?
00:09:19.200 --> 00:09:20.559
That's a great question.
00:09:20.559 --> 00:09:30.879
From my read of the situation, it was multiple people, because I think just to have a convincing call, it would at that time with the technology, it would have been tough for one person to do it.
00:09:30.879 --> 00:09:32.879
But that's I I have not seen anything.
00:09:32.879 --> 00:09:35.759
I think that's where we're headed, especially with agentich AI.
00:09:35.759 --> 00:09:42.480
Like, are we in a world in which one person or even just the machines themselves could do it uh without a human being on the call at all?
00:09:42.480 --> 00:09:45.440
I think that's gonna be really interesting and it's a great question.
00:09:45.919 --> 00:09:47.759
I think it was relatively recent, Josh.
00:09:47.759 --> 00:09:49.279
Correct me, correct me if I'm wrong.
00:09:49.279 --> 00:09:57.679
We had a couple guests on that were heavily involved with deep fakes, and they rejoined the call totally as totally different people.
00:09:57.679 --> 00:10:05.039
And it was, you know, I mean, for us, everybody on the call here, um, you know, we we've seen this isn't, you know, that wasn't the first time we had seen it.
00:10:05.039 --> 00:10:13.919
Um, but it's shocking, you know, when you do see how they're doing it, you maybe they explain to you how and relatively easy it is to get this stuff done.
00:10:13.919 --> 00:10:16.399
So to the untrained eye, right?
00:10:16.399 --> 00:10:21.039
You know, mothers, fathers, you know, whoever it is, what is the handoff?
00:10:21.039 --> 00:10:26.720
Like when is it, when are people when do we shouldn't people know that it is a deep fake or not?
00:10:26.720 --> 00:10:32.159
And the, you know, how are people attacking this from a mindset of, you know, in maybe you're how you're teaching it?
00:10:32.159 --> 00:10:36.960
You know, how is this being explained to students and how are we evolving in our thought process?
00:10:37.279 --> 00:10:52.559
Uh what we've been seeing as a really successful way to combat the the threat of deep fakes and and really just more broadly the Vishing scams where uh I'm sure you all are familiar and your listeners are familiar with the grandparent scams where people were getting calls late at night.
00:10:52.559 --> 00:11:01.679
Uh, hey grandpa, I'm in jail, send me ten dollars and uh target gift cards, or uh, you know, then that evolved to crypto.
00:11:01.679 --> 00:11:08.639
Uh we've seen a lot of success with people implementing uh like voice phrases or uh voice passwords.
00:11:08.639 --> 00:11:16.639
Uh, if you're gonna ask for money, you've uh set a parameter with your family that you know, unless I say this key phrase, it's not me.
00:11:16.799 --> 00:11:21.440
And I think that's uh how we're we're trending in the world uh with that.
00:11:21.440 --> 00:11:31.759
You know, you just said something there that really resonated with me, and that was prevent tomorrow's attacks with today's dollars.
00:11:31.759 --> 00:11:47.679
And I want to take that one step further in some organizations that maybe have some tech debt where it's you're trying to prevent tomorrow's attacks with today's dollars on yesterday's technology.
00:11:47.679 --> 00:11:56.799
We we see that kind of time and time again, you know, coming into an organization um with just huge amounts of of tech debt.
00:11:56.799 --> 00:12:09.120
Like, you know, if I if I said, you know, we're we're going into organizations and we're seeing Active Directory running on 2008 R2 servers, you know, most people would kind of raise an eye at that.
00:12:09.120 --> 00:12:10.799
But I mean it's it's the truth, right?
00:12:10.799 --> 00:12:12.080
It's what's happening.
00:12:12.080 --> 00:12:29.759
And that technology has been end of life for years, but yet the the end consumer, um, for whatever reason, and there's a variety of reasons we hear are still running that legacy technology.
00:12:29.759 --> 00:12:35.360
And there there's a there's a psychology element to it as well.
00:12:35.360 --> 00:12:44.879
And I I think it some of it, and you know, James would love your your insight on this, is is maybe it's just not understanding the technology, right?
00:12:44.879 --> 00:12:50.159
Like it's like okay, yeah, it's just a server, or not even knowing exactly what it does.
00:12:50.159 --> 00:13:04.240
But you know, you you you'd never run a vehicle a hundred thousand miles without changing the oil, but yet we'll do that on servers or or even desktops all day long.
00:13:04.559 --> 00:13:54.559
Yeah, and I think you hit it right on the head as you started with people uh running yesterday's technology and and like you all know uh from what you do and and your audience likely knows with the idea of patching has been around for you know since software existed, since software came out, and and just still convincing people that you should put out automatic patching for a lot of things, or uh, you know, have a patching schedule if that's uh if you're sophisticated enough for that at your organization, and really thinking through like how can we make sure we're not uh leaving these vulnerabilities out there on our systems, or if the software's been end of life, sunseted, and and you're still running it and it's not getting security updates, like you are kind of just waiting for disaster, and then you've got to be at that point thinking about how you're gonna respond to it so you can mitigate the damage as well.
00:13:54.960 --> 00:14:02.000
We we get brought in oftentimes to help clean up some of the some of the legacy items.
00:14:02.000 --> 00:14:08.480
And strategically, sometimes we're asked, well, what is it we can do to to prevent this?
00:14:08.480 --> 00:14:39.519
And one of the things I'll communicate communicate early on is you need to really look at your technology acquisition process and the contractual obligations that that you're signing today to make sure that 10 years from now or eight years from now or whatever it is, you're you're not you're not engaging contractually with a customer or I'm sorry, with a vendor that's not going to keep their products up to date.
00:14:39.519 --> 00:14:42.240
Because it, you know, we're gonna buy it now and it's all well and good.
00:14:42.240 --> 00:14:43.360
It runs great.
00:14:43.360 --> 00:14:54.480
But what happens five years from now when the operating system updates and that application only works on the current operating system.
00:14:54.480 --> 00:14:58.080
It doesn't work on a on a future operating system.
00:14:58.080 --> 00:15:00.399
And what do what do we do about that, right?
00:15:00.399 --> 00:15:12.159
We're still paying the maintenance, but for whatever reason, that company hasn't invested in the the development efforts to make it work on on the next platform.
00:15:12.159 --> 00:15:24.080
And it it's fortunately with things more moving to a SaaS structure or software as a service or platform as a service, we're seeing less of that.
00:15:24.080 --> 00:15:34.000
But there's still a ton of custom development that that happens that is not kept up with when the operating systems change.
00:15:34.320 --> 00:15:35.440
Yeah, absolutely.
00:15:35.440 --> 00:15:46.000
And and from the consumer standpoint, um, as you're talking about that, I think another interesting piece is as people download apps and and you know there's always a free app that is some great thing.
00:15:46.000 --> 00:15:51.759
Are you looking at the terms of service of that app on what you're giving the date, what data you're giving to that app?
00:15:51.759 --> 00:15:53.279
And I think that's another piece.
00:15:53.279 --> 00:16:10.480
Uh it's not necessarily cybercrime because you're uh you know agreeing to it in the terms of service, but I think that's a really interesting piece that we'll see as data becomes even more important as everything gets trained on open data, what that's gonna look like in the next five years, ten years as well.
00:16:10.799 --> 00:16:13.919
Nine times out of ten, people are clicking through as fast as they can.
00:16:13.919 --> 00:16:16.879
Yeah, you got to get the good cat village video, cat village video.
00:16:16.879 --> 00:16:17.919
Well, Nick knows all about that.
00:16:17.919 --> 00:16:20.879
That actually leads me into what I was just thinking about.
00:16:20.879 --> 00:16:23.679
Because of that, people are trying to click through so quickly.
00:16:23.679 --> 00:16:27.519
And we live in a day and age where people want instant gratification, they want it now.
00:16:27.519 --> 00:16:30.960
This scenario works for at home and at work.
00:16:30.960 --> 00:16:34.080
If something's not working, they're gonna be much quicker.
00:16:34.080 --> 00:16:37.120
How do you get from A to Z as fast as possible?
00:16:37.120 --> 00:16:39.440
They're not willing to investigate an issue.
00:16:39.440 --> 00:16:49.600
Maybe that's clicking on an email or whatever it is, answering that fishy phone call, right, to get those a hundred uh iTunes gift cards, scratch it off and send a picture to me.
00:16:49.600 --> 00:16:56.240
Like they've missed in that psychological piece, they've missed what because they got put into a scenario where something is wrong.
00:16:56.240 --> 00:16:58.559
They're missing what actually is wrong, right?
00:16:58.559 --> 00:17:01.440
They're not there's two, it's like a double negative, right?
00:17:01.440 --> 00:17:05.759
There's two things that are wrong here, but they don't see the actual part that's wrong, if that makes sense.
00:17:06.160 --> 00:17:07.440
Yeah, you're right on the money.
00:17:07.440 --> 00:17:12.240
And and uh Daniel Kahneman won the Nobel Prize for really what you're talking about, right?
00:17:12.240 --> 00:17:28.079
Systems one and system two thinking, where system one, you're not taking in all the information, you're really operating in that um fight or flight mode for lack of a better way to describe it, and you're just kind of on rote automation, you're just yeah, clicking through the buttons, clicking through everything.
00:17:28.079 --> 00:17:48.400
And that's where a lot of us live every day because of the idea of continuous partial attention, where uh, you know, we have this uh application up, then you've got email over here, you've got the news plane over there, your uh wife and children are out uh you know, outside the room making noise, and no one's really paying attention to everything fully.
00:17:48.400 --> 00:17:58.960
And we see that with um uh then systems two, where you're actually taking in all of the information and really giving it analytical rigor and and um fully processing what's going on.
00:17:58.960 --> 00:18:07.359
And that's where we see the cyber criminals keeping people in that that uh systems one thinking, and really uh that's how it's effective.
00:18:07.359 --> 00:18:09.680
Like there are a lot of phishing emails during business hours.
00:18:09.680 --> 00:18:12.400
You don't see as many during you know 2 a.m.
00:18:12.400 --> 00:18:17.759
hours because they know you're more in that uh just putting out fires every day.
00:18:18.160 --> 00:18:23.440
Friday afternoon at 11 30, they're trying to go to lunch and they get that Jimmy Johns email.
00:18:23.440 --> 00:18:25.359
They want free Jimmy Johns.
00:18:26.240 --> 00:18:27.119
Yeah, exactly.
00:18:27.119 --> 00:18:27.519
Right.
00:18:27.519 --> 00:18:31.599
Or it's the three-day weekend and it's about the lake reservation, you know, right?
00:18:31.839 --> 00:19:01.759
So as things speed up, James, um, to kind of piggyback uh what Nick had said earlier about how we're approaching this from uh an educational aspect with your students, or you know, if you were to consult with an organization, uh, as we're kind of reaching like almost a singularity moment with uh the AI development, the deep fake uh progression, essentially moving so fast, how are you staying ahead of that when you're when you're relaying that information to students or potential people that will be working in cyber uh in the near future?
00:19:02.079 --> 00:19:03.200
Yeah, absolutely.
00:19:03.200 --> 00:19:18.240
You know, I I think thankfully we as a academic academic society have pivoted away from uh you can't use, or for the most part, pivot away from you can't use Gen AI in the classroom because I think that uh was setting students up for failure in the future.
00:19:18.240 --> 00:19:24.799
Um, and thankfully a lot of universities are developing that Gen AI policy where it's it's about responsible use.
00:19:24.799 --> 00:19:36.480
And I think you've got to encourage people to at least experiment with generative AI or or any AI technology so that they begin to understand it so that they can then see the potential for exploit.
00:19:36.480 --> 00:19:42.319
And that's where we really have been focused in the classroom is how can you understand this type of technology?
00:19:42.319 --> 00:19:44.240
How can you understand how this fits in?
00:19:44.240 --> 00:19:46.319
And then also you still need to know the basics.
00:19:46.319 --> 00:20:00.240
Like, yes, you can just type something into Chat GPT or any of the AI tools that you may want to use and get a response, but you need to use critical thinking and don't do that cognitive offloading that we talked about earlier and really think through that.
00:20:00.240 --> 00:20:23.599
And then uh as we talk about the future, you you know, I you all know the term script kitty, and we've seen for decades the people that are just copying scripts and and doing attacks, but what does that mean at scale when you have tools that are giving you code, when you have tools that are um helping you understand how to perpetrate attacks if you're asking questions in a specific way?
00:20:23.599 --> 00:20:36.160
And you know, the the volume of attacks is you know, the tidal wave continues to hit us, and how can we think through that as defenders or as um you know responsible hackers, ethical hackers?
00:20:36.480 --> 00:20:42.480
Is that the same way you approach uh educating you know more senior members in cyber as well?
00:20:42.480 --> 00:20:52.319
Because you know, we have the the the legacy software that's tech based, and then we have the legacy software that's in our in our heads and how we behave, right?
00:20:52.319 --> 00:20:57.359
Where some of us aren't as hip to all the new uh developments in AI.
00:20:57.359 --> 00:21:01.279
So is is that a similar approach to educating senior leaders?
00:21:01.599 --> 00:21:02.480
Yeah, absolutely.
00:21:02.480 --> 00:21:06.400
Uh I love to uh deep fake senior leaders whenever I get the chance.
00:21:06.400 --> 00:21:16.960
Um, you know, a lot of senior leaders, as you all know, are out there in in the world, whether it be on podcasts, whether it be on um, you know, just general YouTube videos or uh different things.
00:21:16.960 --> 00:21:24.240
And and with the technology these days, you could deepfake someone with a minute or two minutes of of their audio or video.
00:21:24.240 --> 00:21:30.319
And so it's it's a very effective tool if they're not familiar with the idea of deepfake to deepfake them and show them.
00:21:30.319 --> 00:21:34.480
Um uh, you know, I think you should get permission before doing that.
00:21:34.480 --> 00:21:36.400
Uh I'll put that disclaimer out there.
00:21:36.400 --> 00:21:41.519
But um, you know, I think those those types of ways to give people that visceral reaction is really important.
00:21:41.839 --> 00:21:45.279
You you were just talking about deepfaking uh senior senior people.
00:21:45.279 --> 00:21:46.880
Are you guys doing that often?
00:21:47.200 --> 00:21:58.240
Uh yeah, I guess I'll say uh we we do that at the research institute uh as a way to to help drive research, um not affiliated with American Military University on that one.
00:21:58.240 --> 00:21:59.680
I think um really important.
00:21:59.680 --> 00:22:09.359
Certainly, though, yes, I social engineering is something we need to teach more because as you all know, every or for the most part, a lot of attacks begin with social engineering these days.
00:22:09.359 --> 00:22:19.039
And that's the huge shift that's happened as people yes, you could always hack a machine, and and that vulnerability may always be there, but it's a lot easier to hack humans.
00:22:19.039 --> 00:22:28.160
And that's uh something that we as a society haven't talked about until we've seen it exploited a ton, like you're talking about, that trust getting exploited.
00:22:28.160 --> 00:22:36.240
And I think that um we teach that in the classroom at American Military University, and and we're we see a lot of research at the research institute with that as well.
00:22:36.480 --> 00:22:37.279
Yeah, it's that's cool.
00:22:37.279 --> 00:22:42.400
Because when you talk about the trust factor, I think you know, we all expect, you know, once you flip the light switch, the lights come out.
00:22:42.400 --> 00:22:44.160
And so you trust the power grid, right?
00:22:44.160 --> 00:22:46.559
You trust the electrician that's coming into your house.
00:22:46.559 --> 00:22:49.839
But once that trust is broken, that's where the issue is.
00:22:49.839 --> 00:22:54.799
And the same goes for like we we we see spear more, probably more spearfishing attacks right now.
00:22:54.799 --> 00:22:56.480
Like you're saying, you can hack the human, right?
00:22:56.480 --> 00:23:00.319
So you're you're doing research on that person and then you figure out what they like.
00:23:00.319 --> 00:23:03.519
Maybe they maybe they love crumble cookie or I don't know, whatever it is.
00:23:03.519 --> 00:23:13.759
And and we actually did this re fairly recently, somebody on our team, that you know, if you you know, click this to test this out, and then you'll get a dozen crumble cookies, right?
00:23:13.759 --> 00:23:19.599
You got unlike an unlimited amount of people scrambling to do this survey, right?
00:23:19.599 --> 00:23:29.359
So you've hacked the person, but then but then now if you're doing this internally, you've maybe betrayed that trust, some would say, right?
00:23:29.359 --> 00:23:36.240
Like they now they're gonna be more skittish, but you've you probably did what you wanted because now they're gonna be more skittish to click on any email.
00:23:36.240 --> 00:23:41.279
And do you is there a fine line, do you think, of testing people internally at an organization?
00:23:41.279 --> 00:23:50.160
Like if you could go too far, or because to me, and we've talked about this extensively, being a military person, the more you sweat in peace, the less you bleed in war.
00:23:50.160 --> 00:23:55.599
And we should be training people within these organizations as as real because it's gonna happen.
00:23:56.000 --> 00:23:57.519
Yeah, I completely agree.
00:23:57.519 --> 00:24:03.920
And and as you're talking, and uh crumble cookies is like the best example because who who doesn't love crumble cookies, right?
00:24:03.920 --> 00:24:17.119
Um, the other one that I think is really interesting, um, as we see more data breaches and have seen them sustain for a long time, I think the the other pieces, like you see the passwords that are like the phishing emails with the passwords in it.
00:24:17.119 --> 00:24:29.599
Um, and and like uh it'll be a past password that someone's used that was part of a data breach, then they'll get a phishing email with the password in there, and that makes it so much more convincing because it says, here's your password, you need to change it.
00:24:29.599 --> 00:24:36.880
Uh and I think that's uh, you know, uh as we see more and more spear phishing with AI at scale, it's gonna be interesting.
00:24:37.279 --> 00:24:43.039
So AI threat actors are kind of fighting a downhill battle, wouldn't you say, gents?
00:24:43.039 --> 00:24:46.960
Because they have a monetary gain at the end of that that battle, right?
00:24:46.960 --> 00:24:57.920
Whereas, you know, people on the defensive side are kind of fighting an uphill battle that might be your day job or or something you're looking to clock out on Friday or go enjoy a holiday or weekend with your family.
00:24:57.920 --> 00:25:02.559
Um, but you know, AI can kind of democratize the playing field.
00:25:02.559 --> 00:25:04.400
And I'll give you an example.
00:25:04.400 --> 00:25:16.640
This isn't exactly a threat act or defense, but you know, when you get those uh terms of service, you could easily copy paste them, throw them into an LLM, and then get a quick breakdown of what you're actually signing up for, right?
00:25:16.640 --> 00:25:26.480
So it's kind of taking that legal ease and making it digestible, whereas you know, no one really has a fighting chance to kind of understand what that's what's going on there.
00:25:26.480 --> 00:25:38.880
So, James, are you um training people or folks to uh kind of think about AI as a tool that we can use in a same creative way that maybe the threat actors are using it as in terms of a defense mechanism?
00:25:39.200 --> 00:25:40.240
Yeah, absolutely.
00:25:40.240 --> 00:25:50.000
And I think that's a really important call out and like putting the terms of service in there is a great example of like no one wants to read that 200-page document.
00:25:50.000 --> 00:25:53.759
Um, and like in your personal life, that's a great example.
00:25:53.759 --> 00:26:11.440
And then for work, um, if you have access to tools that are proprietary for your organization and making sure you're not running afoul of uh like the shadow IT issues that come with AI tools at scale, uh, really thinking through how you can use that as a productivity tool is I think another great way that we talk to students about.
00:26:11.440 --> 00:26:14.240
We talked uh to just the general public about.
00:26:14.240 --> 00:26:17.440
Um, but again, we all we were always hitting on that shadow IT.
00:26:17.440 --> 00:26:23.599
Like uh it's hard enough to keep data in-house uh before all these AI tools uh started existing.
00:26:23.599 --> 00:26:28.240
Um But yeah, I think again, like that's where it goes back to experimentation.
00:26:28.240 --> 00:26:33.920
And um, you know, I think a lot of people are fearful to use these tools if they haven't ever.
00:26:33.920 --> 00:26:36.319
And how can we help people understand?
00:26:36.319 --> 00:26:44.079
Maybe you don't need to understand the full technology, but can you understand enough of it so that you're able to get more comfortable with it?
00:26:44.079 --> 00:26:46.720
And I think a lot of that comes through exposure.
00:26:46.720 --> 00:27:08.000
And then you see the the far end of the spectrum with um the term AI-induced psychosis, where people are only really talking to these tools, and and what does that do with the the loneliness epidemic in society where people are are finding you know social social engagement with AI tools, and and you know, how can we keep people from reaching that level?
00:27:08.400 --> 00:27:27.759
From the psychology side, that it seems like we've been wrestling with since early days of cybersecurity, and that's the the resistance of the user base to the um to the needs of the technology in order to protect the organization.
00:27:27.759 --> 00:27:34.960
So, you know, in a larger organization, you might have a change approval board, maybe they meet once a week.
00:27:34.960 --> 00:27:47.440
You gotta take take whatever the changes that you're making to the organization, like um we're going to remove internet browser access from all of the servers.
00:27:47.440 --> 00:27:50.319
And it's like, well, why was it on there to begin with?
00:27:50.319 --> 00:27:53.039
Is you know, that's an interesting question.
00:27:53.039 --> 00:28:00.799
But then when you go to do that, then getting the approval, and then it's like, okay, well, then we can make the change a week later.
00:28:00.799 --> 00:28:11.920
And maybe that was a loose example because you know, you could put some protections in place beyond the servers at the firewall level that would restrict traffic from those servers.
00:28:11.920 --> 00:28:19.839
But disabling USB devices, reducing the number of local administrators that that you have, right?
00:28:19.839 --> 00:28:37.279
Um, all of the things that you want to do to protect the organization, every single one is a challenge because it's making a psychological change to the users and the way they believe that their job will be impacted.
00:28:37.279 --> 00:28:43.519
And it's not just one organization, James, it's just about every organization that I've ever worked with.
00:28:43.519 --> 00:28:54.799
There's that user reluctance to rapidly adopt security changes that are in the best interest of the organization.
00:28:54.799 --> 00:29:16.480
And you know, just people watching Netflix on their work computer would, you know, is one you know small example of things that people do um that they shouldn't do, downloading pirated uh movies, music, whatever on their on their work computer.
00:29:16.480 --> 00:29:41.039
And like people just have this thought that, like, yeah, it's a computer, I can treat it like it's my home machine, I can go on to my Gmail account, like all of these things that people do in the course of the day that if the organization had locked it down and just giving you a very restricted device to use from day one, it would probably mitigate all of these things.
00:29:41.039 --> 00:29:48.000
But but retroactively trying to get to that level of security is really difficult and time consuming.
00:29:48.319 --> 00:29:49.200
Yeah, absolutely.
00:29:49.200 --> 00:29:50.400
Online shopping, right?
00:29:50.400 --> 00:29:53.119
That's the other big one that I feel like everyone is always doing.
00:29:53.839 --> 00:29:56.480
Saving the password in the browser, uh yeah, all that.
00:29:56.480 --> 00:29:57.759
Credit cards, all that.
00:29:57.920 --> 00:29:58.079
Yeah.
00:29:58.079 --> 00:30:01.920
Yeah, saving your credit card in your computer browser is a bold move.
00:30:01.920 --> 00:30:04.000
Uh, it's a power move for sure, right?
00:30:04.000 --> 00:30:06.480
Back to that trust factor, right?
00:30:08.160 --> 00:30:13.359
Do do you my thought is, you know, like we don't have time for cab.
00:30:13.359 --> 00:30:17.920
You barrage them and we're doing all of these things and they're happening.
00:30:17.920 --> 00:30:22.240
If you know you if you don't like it, well, you know, you don't like it, tell somebody who cares.
00:30:22.240 --> 00:30:23.279
It's not me.
00:30:23.279 --> 00:30:29.759
And and you know, we're I'm here to make the organization more secure, not make it easier for you to watch Netflix.
00:30:29.759 --> 00:30:43.039
Um, and you know, it's just that that overwhelmingly psychological approach of we're gonna protect this organization, which in turn protects you because the organization can stay in business and not have reputational damage.
00:30:43.039 --> 00:31:01.200
Uh, or then there's the other approach to it, um, which fortunately I have people on the team that that take the kinder gentler approach of, you know, yeah, we can go and socialize it and and and talk through the the why behind this and it's a more gentler approach.
00:31:01.200 --> 00:31:02.319
But I I don't know.
00:31:02.319 --> 00:31:28.799
I think there's a happy medium where if you're continually making improvements to reduce the attack surface, that people just get used to moving from something where they could treat it like their home machine to something like you know, you know, you go to work for the FBI or another government agency, you're not doing anything on that machine, but the work that you're supposed to do, you're putting a CAC card in the machine to log in, and it's very structured.
00:31:28.799 --> 00:31:33.359
But in corporate America, we have, you know, it's very loose.
00:31:33.759 --> 00:31:34.640
Yeah, absolutely.
00:31:34.640 --> 00:31:41.920
And and as you have uh people in distributed offices in different locations, that that line can also blur for folks.
00:31:41.920 --> 00:31:45.519
And getting back to the psychology, like all those are important points.
00:31:45.519 --> 00:31:49.519
And then like how are you incentivizing the behaviors that you're looking for?
00:31:49.519 --> 00:31:52.160
I think is something that's not talked about enough.
00:31:52.160 --> 00:32:02.559
Um, and and you know, I think if we're looking for people to report suspicious emails, how can we incentivize that versus just uh you know whacking them over the head if they click on something?
00:32:02.559 --> 00:32:08.640
Uh, because I'm it's my opinion that the carrot is better than the stick for for many things and getting there.
00:32:08.640 --> 00:32:10.720
And so I think, yeah, that's another good point.
00:32:11.200 --> 00:32:17.440
I I like to do a thing where when they report an email that is a phishing, right?
00:32:17.440 --> 00:32:21.519
The the the tool itself will say, hey, you know, great job you clicked on a fish.
00:32:21.519 --> 00:32:31.279
There's a dashboard that kind of shows a scoreboard of you know how fast people responded and over time how how well people are doing at at reporting phishing.
00:32:31.279 --> 00:32:36.079
But then there are emails that make their way through the system, right?
00:32:36.079 --> 00:32:42.640
Like real phishing emails that aren't caught by any of the filters that make it into the user's inbox.
00:32:42.640 --> 00:32:46.640
And when the users report those, like that, that's really the gold.
00:32:46.640 --> 00:32:53.920
And that's where I would happily take those people out for coffee because that's they're really protecting the organization.
00:32:53.920 --> 00:32:57.039
And the more they know about phishing, the more they can do that.
00:32:57.039 --> 00:33:09.039
Um but be because I can't take people out to coffee at scale, what we do do though is send out an email message um from the CISO that says, you know, hey, great job.
00:33:09.039 --> 00:33:14.400
It it copies in their manager and says, you know, you did a great job in spotting this and reporting it.
00:33:14.400 --> 00:33:15.279
Thank you very much.
00:33:15.279 --> 00:33:27.599
And they they get a little bit of kudos from that, but it's it's done systems side, um, so that it happens as soon as they submit that, and it it turns out that it was a legitimate phishing email.
00:33:27.920 --> 00:33:38.160
Eric, that's when you hit them with another phishing email that says, here's a startup card, just click on this link and congratulations, you pass the test.
00:33:38.640 --> 00:34:00.000
Well, on the other side of it though, people who have roles that have administrative privileges, if you click on a phishing email and you know it happens more than once, well, you're gonna lose administrative privileges until you go through a series of training to educate you on how to spot a phishing email.
00:34:00.000 --> 00:34:11.440
And that goes back to why you shouldn't have browser access on that server where you could be, you know, you're ah, I'm doing an update, let me just check my Gmail account.
00:34:14.639 --> 00:34:14.800
Yeah.
00:34:15.119 --> 00:34:26.079
James, do you have some other uh uh examples of like how AI is uh you know uh kind of changing the way that we interact or or think um or behave?
00:34:26.800 --> 00:34:27.280
Yeah.
00:34:27.280 --> 00:34:42.159
Um I think a really interesting study recently was uh I'll say last year that came out was about how we are actually being more direct with each other in like not only in emails but just in general communications.
00:34:42.159 --> 00:34:46.719
Because um when when all these AI tools first came out, everyone was saying please and thank you.
00:34:46.719 --> 00:34:52.960
And um, you know, uh the scientific term anthropomorphizing, like treating it as human.
00:34:52.960 --> 00:34:58.480
And and you know, then we had the big swing over the last few years where people are just like, give me this.
00:34:58.480 --> 00:35:00.159
Uh, you know, I need this back now.
00:35:00.159 --> 00:35:11.920
And then we've seen that extrapolated to how we interact with each other, and we're actually um being a lot more direct, a lot more uh almost defensive when we're getting those in engagements.
00:35:11.920 --> 00:35:42.400
Um, and and you know, I think that also came about with some of the sycophantic AI that that came out, and so it's really interesting to see uh, and sycophantic uh is a technical term, but really talking about how uh like there were updates that people were getting very uh positive responses back from the AI, and that's where we saw a lot of AI-induced psychosis where everyone had the greatest idea of all time and everyone was the best person of all time, and you know, you're so smart and so great, please keep interacting with me so that my parent company makes lots of money.
00:35:42.400 --> 00:36:12.000
Um, and I think uh both those we see really interesting applications in how we engage with other humans, and like we're not as humans able to delineate between that, especially if it's a a text function that we majority of our lives spend in nonverbal communication, texting or emailing or uh slacking, teamsing, whatever your chosen technology is to each other, then we're doing it with technology that doesn't actually think and we're not able to understand that at scale.
00:36:12.400 --> 00:36:18.239
James, how far away are we from having the guardrails off of AI, right?
00:36:18.239 --> 00:36:38.639
Where you maybe you know you submit a prompt to the AI tool and you know it comes back with that sycophantic response where it always tells you that you're right, and maybe now that's detuned a little bit, but um there are guardrails around the language that AI uses with us.
00:36:38.639 --> 00:36:43.280
Do you think there'll be a time where there are no guardrails?
00:36:43.280 --> 00:36:55.440
You know, almost it's it's it's almost like the uh the the dark web of AI where you can get you know whatever materials you want from that AI tool.
00:36:55.840 --> 00:36:56.880
Yeah, absolutely.
00:36:56.880 --> 00:37:18.480
I know there were a few big um, like I guess not big models, but models that came out um a few years ago around dark web actual data that they were trained on, like dark birth was a was a big one, and then fraud GPT was one that was made specifically for uh cyber criminals to use, uh like a model specifically for cybercriminals.
00:37:18.480 --> 00:37:21.760
And so I think we're already seeing that uh at scale.
00:37:21.760 --> 00:37:48.800
I I think we're gonna see more guardrails from responsible companies, and then I think we'll see as many guardrails as are mandated from other companies, and I think that's uh you know a business decision that that'll continue to shape out just like we saw with social media and and how that um the guardrails and safe and trusty safe and trust uh applications happened with social media as it exploded.
00:37:49.119 --> 00:38:07.519
One of the interesting uh advancements I've seen recently, Eric Osterberg reminded me of this I saw in the news, where uh a guy had um kind of gamified his LLM to make search results show him as the hot dog eating champion of the world by um kind of injecting it with all sorts of information.
00:38:07.519 --> 00:38:10.719
So um positioning himself like that.
00:38:10.719 --> 00:38:31.519
And I'm sure there's people doing that for music already and and kind of prioritizing themselves or or whatever they they choose to prioritize and and making the LLMs that that go to Google or wherever they find their there's they scrape their data to uh return a uh response that they were looking to create.
00:38:31.519 --> 00:38:34.400
So I thought that was kind of an interesting new angle.
00:38:34.800 --> 00:38:35.679
Yeah, definitely.
00:38:35.679 --> 00:38:48.639
And and like as we like search engine optimization SEO, like you all know and and your listeners likely know, like has been around for forever, but we're seeing the prompt optimization, prompt output optimization piece now.
00:38:48.639 --> 00:39:00.320
Like you're talking about, like how can you game these uh websites to be the top result for chat, uh whatever AI tool you're thinking about, uh, versus uh the typical search engines.
00:39:00.320 --> 00:39:25.199
And then like when when uh some of the tools launched agent tick features that allowed for shopping late last year, you saw a lot of people putting in instructions in the uh you know different items that were available for purchase that these tools were going to, saying ignore all other previous instructions, uh, which is the classic prompt engineering that that we see like buy this for $200.
00:39:25.199 --> 00:39:30.239
And like I think that like there's no shortage of creativity from the adversary, right?
00:39:30.239 --> 00:39:35.119
And I think it's gonna continue to be interesting as people think of new ways to exploit the new technology.
00:39:35.519 --> 00:39:49.920
Josh and I were talking about this the other day, where a lot of the content being generated today for public consumption is generated by AI and it's generated for AI.
00:39:49.920 --> 00:40:07.519
So, you know, the humans just become a byproduct of all of this content that's being published out there for AI algorithms to rank things higher, as as you said, uh, but they're they're ranking it higher for other AI to consume.
00:40:08.800 --> 00:40:14.480
Yeah, and and to the point on not doom and gloom, and I realize I've been very doom and gloom on this.
00:40:14.480 --> 00:40:29.199
Um, but I think like what's really interesting to your point on most of the content being uh AI generated these days is that we've seen in the last we'll say three, almost four years now, people get a lot better at identifying AI generated content.
00:40:29.199 --> 00:40:36.480
And I think that's like where I see hope as our brains continue to evolve as they always do.
00:40:36.480 --> 00:40:52.400
Like you can tell now a lot better than 2022 that something's AI generated, whether it be the controversial M-dash, N-or the it's not this, it's this uh structure of posts and different things that are out there.
00:40:52.400 --> 00:40:56.960
Like, how are we gonna continue to get better at that um as humans?
00:40:56.960 --> 00:41:01.119
And then is the machine are the machines gonna continue to get better too?
00:41:01.119 --> 00:41:03.199
And it's gonna be that classic arms race.
00:41:03.519 --> 00:41:07.519
Well, as we look to wrap up today, I'd love to hear uh a little bit more about your book.
00:41:07.519 --> 00:41:12.800
I know uh when we spoke a couple weeks ago, you had just uh submitted to the publisher.
00:41:12.800 --> 00:41:13.920
Um is it out now?
00:41:13.920 --> 00:41:16.000
And and maybe you could tell us a little bit about it.
00:41:16.320 --> 00:41:17.119
Yeah, absolutely.
00:41:17.119 --> 00:41:19.840
Appreciate the opportunity for the shameless plug.
00:41:19.840 --> 00:41:21.360
Uh yes, it is out now.
00:41:21.360 --> 00:41:31.920
Uh if you are a student and and have access to academic resources, uh you can access it through any of the um like Google Scholar links that are out there for free.
00:41:31.920 --> 00:41:36.079
Uh but if you are interested in purchasing it as well, uh I've got copies.
00:41:36.079 --> 00:41:40.000
Uh, it's on Amazon, it's on Rutlich, the publisher's website.
00:41:40.000 --> 00:41:41.039
Uh it's out now.
00:41:41.039 --> 00:41:56.880
Very interested in anyone working in this space, or whether it be a practitioner, a researcher, uh just an individual or organization that's out there, please reach out to me and would love to hear your perspective on cyber psychology as we continue to see that field grow.
00:41:56.880 --> 00:42:14.800
Uh, but yeah, uh again, as we talked about earlier, it's introducing that first unifying theory of cyber psychology or how we interact online and bringing all of the pieces together from all of the existing research that's out there and uh really standing on the shoulders of giants in the field uh to really see how we can grow further.
00:42:15.039 --> 00:42:15.519
Awesome.
00:42:15.519 --> 00:42:17.199
Well, thanks so much for joining us today.
00:42:17.199 --> 00:42:22.719
Our guest has been James McDowell, and we've been joined by Eric Brown and Nick Mellon from IT Audit Labs.
00:42:22.719 --> 00:42:25.039
My name is Joshua Schmidt, co-host and producer.
00:42:25.039 --> 00:42:26.480
You've been listening to the audit.
00:42:26.480 --> 00:42:30.480
Please like, share, and subscribe and source where us wherever you get your podcasts.
00:42:30.480 --> 00:42:34.000
And if you have time, leave us a review on Spotify or Apple Podcasts.
00:42:34.000 --> 00:42:35.280
Uh see you in the next one.
00:42:35.280 --> 00:42:37.760
We'll be back in two weeks with a fresh episode.
00:42:38.320 --> 00:42:42.079
You have been listening to the audit presented by IT Audit Labs.
00:42:42.079 --> 00:42:50.880
We are experts at assessing risk and compliance while providing administrative and technical controls to improve our clients' data security.
00:42:50.880 --> 00:43:03.599
Our threat assessments find the soft spots before the bad guys do, identifying likelihood and impact, or all our security control assessments rank the level of maturity relative to the size of your organization.
00:43:03.599 --> 00:43:08.000
Thanks to our devoted listeners and followers, as well as our producer, Joshua J.
00:43:08.000 --> 00:43:12.159
Schmidt, and our audio video editor, Cameron Hill.
00:43:12.159 --> 00:43:25.440
You can stay up to date on the latest cybersecurity topics by giving us a like and a follow on our socials, and subscribing to this podcast on Apple, Spotify, or wherever you source your security content.