Analyst Talk - Before You Leap - Navigate AI Without Losing Tradecraft

1
00:00:00,000 --> 00:00:01,830
Welcome to Analyst Talk with Jason Elder.
2
00:00:01,920 --> 00:00:04,800
It's like coffee with an analyst,
or it could be whiskey with an
3
00:00:04,800 --> 00:00:08,070
analyst reading a spreadsheet,
linking crime events, identifying a
4
00:00:08,070 --> 00:00:11,640
series, and getting the latest scoop
on association news and training.
5
00:00:11,730 --> 00:00:15,150
So please don't beat that analyst and
join us as we define the law enforcement
6
00:00:15,150 --> 00:00:17,490
analysis profession one episode at time.
7
00:00:18,400 --> 00:00:18,999
How we doing?
8
00:00:18,999 --> 00:00:23,709
Analyst, Jason Elder here with
another LE, a podcast Deep Dive
9
00:00:23,830 --> 00:00:26,409
before you leap with Randy Stickley.
10
00:00:26,409 --> 00:00:27,310
Randy, how we doing?
11
00:00:27,430 --> 00:00:28,540
I am good, man.
12
00:00:28,540 --> 00:00:29,560
Happy New Year.
13
00:00:29,619 --> 00:00:31,990
Happy New Year, indeed.
14
00:00:32,049 --> 00:00:34,360
2026, man.
15
00:00:34,420 --> 00:00:36,970
Yeah, I thought we would have so
many things figured out by now.
16
00:00:37,150 --> 00:00:37,474
I mean.
17
00:00:37,995 --> 00:00:41,894
I think we figured out a few, but we,
we definitely made some more messes.
18
00:00:44,384 --> 00:00:45,015
Oh man.
19
00:00:45,015 --> 00:00:47,535
So are you a New Year's resolution guy?
20
00:00:48,294 --> 00:00:49,824
Typically I am, but mm-hmm.
21
00:00:50,124 --> 00:00:53,004
My New Year's resolutions
are a little more esoteric.
22
00:00:53,065 --> 00:00:57,335
And again, that's kind of on brand
for me, speaking on this podcast,
23
00:00:57,425 --> 00:01:01,764
a lot of mine are pontification
internal kind of reflection.
24
00:01:02,105 --> 00:01:05,224
As opposed to physical things,
because what am I gonna tell myself?
25
00:01:05,585 --> 00:01:06,995
Go to the gym more often?
26
00:01:06,995 --> 00:01:09,514
Well, my God, I go there enough so.
27
00:01:09,875 --> 00:01:12,995
It's a lot of like, self-help kind
of things that, that sort of deal.
28
00:01:12,995 --> 00:01:13,625
What about you?
29
00:01:13,695 --> 00:01:14,865
Speaking of the gym.
30
00:01:15,134 --> 00:01:18,014
I started a month early in December.
31
00:01:18,014 --> 00:01:21,255
I have joined my wife
and going to the gym.
32
00:01:21,255 --> 00:01:28,545
I, my wife is probably would be
considered a gym rat and I have joined
33
00:01:28,905 --> 00:01:34,815
her and she enjoys kicking my butt
a couple of days a week at the gym.
34
00:01:35,465 --> 00:01:40,885
I will tell you my wife and I during
COVID started going to the gym together
35
00:01:41,005 --> 00:01:45,415
and I found we have two completely
different styles of working out
36
00:01:45,505 --> 00:01:50,545
because I was Mr. Contact sports
and my wife was as well, but her big
37
00:01:50,545 --> 00:01:51,985
bread and butter was always swimming.
38
00:01:53,540 --> 00:01:55,370
I mean, she was a, a prolific swimmer.
39
00:01:55,370 --> 00:02:00,300
We were in high school and we joined this
new gym over in Northern Virginia and
40
00:02:00,300 --> 00:02:03,330
she would come lift with me every now
and then, and I would be the guy that
41
00:02:03,330 --> 00:02:05,160
would go, come on, you got one more?
42
00:02:05,160 --> 00:02:06,180
You got one more.
43
00:02:06,180 --> 00:02:07,770
And I would get the look.
44
00:02:07,860 --> 00:02:12,120
So then I would go join her in the lap
pool and I would start swimming and she
45
00:02:12,120 --> 00:02:14,040
would go, yeah, you're not kicking right.
46
00:02:15,060 --> 00:02:17,010
And I was like, wow, that's
what that feels like.
47
00:02:17,880 --> 00:02:20,640
So we have our clear lines
of where our lanes are, but
48
00:02:20,640 --> 00:02:22,050
it's, it's been great for us.
49
00:02:22,155 --> 00:02:22,445
Yeah.
50
00:02:22,770 --> 00:02:23,130
Yeah.
51
00:02:23,190 --> 00:02:29,440
So my, my wife is also a swimmer and so
she makes fun of my lack of flexibility.
52
00:02:31,490 --> 00:02:32,870
That's what wives are for though.
53
00:02:32,870 --> 00:02:34,910
They, they just love to, to bust up on us.
54
00:02:34,910 --> 00:02:35,870
Man, it's good.
55
00:02:35,870 --> 00:02:36,950
We love, we love each other.
56
00:02:36,950 --> 00:02:37,550
It's all good.
57
00:02:38,720 --> 00:02:38,730
Yeah.
58
00:02:38,730 --> 00:02:39,080
Yeah.
59
00:02:39,080 --> 00:02:40,210
So , very good.
60
00:02:40,210 --> 00:02:41,650
What are we talking about today?
61
00:02:41,770 --> 00:02:46,180
We're gonna stick, obviously with the
motif of critical thinking and intel
62
00:02:46,180 --> 00:02:50,740
work, but I think one of the biggest
topics that's going to impact is
63
00:02:50,740 --> 00:02:52,930
impacting and will continue to impact.
64
00:02:53,650 --> 00:02:58,975
The idea of critical thinking at Intel is
the the dreaded dun, dun, dun, artificial
65
00:02:58,975 --> 00:03:03,475
intelligence, AI learning language
models, LLMs, whatever you wanna call
66
00:03:03,475 --> 00:03:09,105
it in their use and kind of loose nature
in the environment that we work in.
67
00:03:09,255 --> 00:03:13,005
I, I've mentioned this to
conversations with colleagues,
68
00:03:13,005 --> 00:03:17,745
peers, and even folks that are in,
in academia at this point, but AI.
69
00:03:18,010 --> 00:03:19,270
Isn't going anywhere.
70
00:03:19,360 --> 00:03:25,430
I like to use the analogy that AI is a
rising tide, that quite honestly, everyone
71
00:03:25,430 --> 00:03:28,280
in some way, shape, or form is using it.
72
00:03:28,280 --> 00:03:31,010
I've said the, the very same thing
about critical thinking, whether
73
00:03:31,010 --> 00:03:32,780
you realize you're using it or not.
74
00:03:33,105 --> 00:03:34,725
You're employing critical thought.
75
00:03:34,725 --> 00:03:40,365
Well, AI has become such a proliferated
resource, tool, software, whatever you
76
00:03:40,365 --> 00:03:43,365
want to brand it as in our common space.
77
00:03:43,775 --> 00:03:46,595
I mean things as simple as shopping apps.
78
00:03:46,595 --> 00:03:49,510
That little AI assistant that
pops up and asks you hey.
79
00:03:50,175 --> 00:03:51,375
Can I help you with anything?
80
00:03:51,375 --> 00:03:52,995
Can I help you find something?
81
00:03:53,145 --> 00:03:58,155
Going a step beyond that, though, you
have such a, a wide array of tools.
82
00:03:58,155 --> 00:04:02,395
Copilot, Gemini, the GPTs, I
mean, whatever you want whatever,
83
00:04:02,395 --> 00:04:03,865
whichever one your flavor is.
84
00:04:03,955 --> 00:04:09,260
They're typically used for varying degrees
of utility in our personal lives, whether
85
00:04:09,290 --> 00:04:13,705
it's augmentation of thought processes,
kind of using it as a soundboard.
86
00:04:14,330 --> 00:04:15,170
Planning.
87
00:04:15,230 --> 00:04:19,460
I know many folks who kind of
put their daily habits into a GPT
88
00:04:19,460 --> 00:04:23,299
wrapper and let it give a, here's
what you should do to maximize
89
00:04:23,299 --> 00:04:25,340
the efficiency of your day aspect.
90
00:04:25,429 --> 00:04:25,520
Mm-hmm.
91
00:04:25,760 --> 00:04:27,650
Research, I'm guilty of this.
92
00:04:27,650 --> 00:04:31,250
Whenever I hit a wall, whenever
I hit a certain point, when I was
93
00:04:31,300 --> 00:04:35,525
working on the book cri the critical
thinking book that KU and I wrote.
94
00:04:36,025 --> 00:04:39,025
I use it as a research soundboard
just to make sure that I was in a
95
00:04:39,025 --> 00:04:42,775
good spot or even flat out execution.
96
00:04:42,775 --> 00:04:45,355
And when I say execution, I don't
mean it in a terminal sense.
97
00:04:45,355 --> 00:04:50,845
I mean it in a sense of there are folks
that plug all the data that they have
98
00:04:50,845 --> 00:04:54,805
or all the resources they have available
for whatever the topic may be, and
99
00:04:54,805 --> 00:04:59,284
they just ask an AI model or a large
language model, learn language model.
100
00:04:59,624 --> 00:05:03,705
To produce some type of tangible
and actionable output for them.
101
00:05:03,794 --> 00:05:07,754
That's where I start to get a
little bit of a hesitation about me.
102
00:05:07,754 --> 00:05:10,994
I don't know if you and your
conversations before have had con this.
103
00:05:11,360 --> 00:05:16,790
Air from folks, but folks using
AI platforms and AI software to
104
00:05:16,790 --> 00:05:19,010
flat out do their tasks for them.
105
00:05:19,010 --> 00:05:23,000
Especially when it comes to something
like writing an academic paper or
106
00:05:23,000 --> 00:05:29,930
writing a memo email, or even in some
cases a tangible analytic product.
107
00:05:30,020 --> 00:05:34,229
It, it tends to get a little bit of
a. A spidey sense tingling in my book.
108
00:05:34,229 --> 00:05:36,900
Have you gathered that from
some of your prior folks, Jason?
109
00:05:37,049 --> 00:05:40,770
Well, I, I think it's, the answer is yes.
110
00:05:40,830 --> 00:05:46,319
I do feel though, that it's, it's
always interesting to define the terms
111
00:05:46,319 --> 00:05:54,150
and define what it's used for and just
what aspect of AI you're relying on.
112
00:05:54,210 --> 00:05:56,099
'cause it's, it's interesting.
113
00:05:56,159 --> 00:05:57,489
I had Ryan Kapaun on.
114
00:05:58,640 --> 00:06:05,669
And I asked him, I said, Hey, if one
of your interns used chat GPT to edit a
115
00:06:05,669 --> 00:06:07,530
bulletin, how would you feel about that?
116
00:06:07,710 --> 00:06:07,800
Mm-hmm.
117
00:06:08,070 --> 00:06:11,430
And he kinda started off as saying
like, I'd be super disappointed.
118
00:06:11,430 --> 00:06:16,059
And then it kind of softened his blow
from there as we talked about it.
119
00:06:16,179 --> 00:06:20,619
From people that are really
good writers, they really scoff.
120
00:06:20,919 --> 00:06:26,829
At the idea of using chat GPT to
edit your writing, but for folks that
121
00:06:26,829 --> 00:06:32,359
aren't so good at like writing like I
am it's, it's a breath of fresh air.
122
00:06:32,419 --> 00:06:32,869
Yeah.
123
00:06:32,869 --> 00:06:36,499
I mean, don't get me wrong, that's where
I think the the sound boarding aspect
124
00:06:36,499 --> 00:06:38,839
comes into play that I referenced earlier.
125
00:06:38,839 --> 00:06:41,879
But something that I'll, and
it's a brief dive tribe of mine.
126
00:06:42,189 --> 00:06:47,349
I've seen and actually read news articles
where professional organizations and
127
00:06:47,349 --> 00:06:53,199
government agencies are now employing
AI to write either civil service
128
00:06:53,199 --> 00:06:58,269
products that are going out to a wide
stakeholder base, or consulting firms
129
00:06:58,269 --> 00:07:02,379
that are actually working and using
AI to build their tangible outputs
130
00:07:02,379 --> 00:07:06,279
to their clients that millions of
dollars may be poured into these
131
00:07:06,279 --> 00:07:08,859
contracts and these consulting fees and.
132
00:07:09,214 --> 00:07:13,654
We're gonna hit on something here in a
moment, but there's a problem with that.
133
00:07:13,714 --> 00:07:17,974
One of my biggest things, and it's a
red flag for me, and I saw this, and I
134
00:07:17,974 --> 00:07:21,924
won't say the agency because I don't want
to, get any dirty stares from any of my
135
00:07:21,974 --> 00:07:24,284
partners and colleagues that may do this.
136
00:07:24,494 --> 00:07:24,584
Mm-hmm.
137
00:07:25,064 --> 00:07:28,904
But I was reading an assessment
following the the extradition of
138
00:07:28,904 --> 00:07:30,884
Nicholas Maduro from Venezuela.
139
00:07:30,974 --> 00:07:33,944
We're all probably aware of that
happening in the news here recently.
140
00:07:34,094 --> 00:07:38,144
This agency, I was reading their
brief on the scenario and the threats
141
00:07:38,144 --> 00:07:40,154
therein, kind of in the fallout.
142
00:07:40,334 --> 00:07:46,054
And I could tell blatantly that it was
written with an AI platform, A GPT,
143
00:07:46,054 --> 00:07:47,824
whatever, whichever the flavor is.
144
00:07:48,604 --> 00:07:48,694
Mm-hmm.
145
00:07:48,934 --> 00:07:48,974
Like I said earlier.
146
00:07:48,974 --> 00:07:48,984
Mm-hmm.
147
00:07:49,024 --> 00:07:50,494
And it's actually cited.
148
00:07:50,874 --> 00:07:52,044
That it was written with that.
149
00:07:52,044 --> 00:07:52,134
Mm-hmm.
150
00:07:52,464 --> 00:07:54,294
I didn't even see the citation.
151
00:07:54,294 --> 00:07:55,914
I just could read it.
152
00:07:55,914 --> 00:07:59,694
And having seen enough of these
outputs, I know what one of the
153
00:07:59,694 --> 00:08:03,954
biggest red flags is, which I'm not
sure if many listeners are aware.
154
00:08:04,844 --> 00:08:09,894
But one of the biggest dead giveaways
whenever you're writing or I shouldn't
155
00:08:09,894 --> 00:08:14,484
say writing, but like reading a product or
reading an output of some kind to see if
156
00:08:14,484 --> 00:08:19,494
it was written with a GPT or something, is
checking for something called the M dash.
157
00:08:19,644 --> 00:08:21,834
Are you familiar with the MM dash Jason?
158
00:08:21,954 --> 00:08:22,404
I am.
159
00:08:22,524 --> 00:08:24,834
It's one of my favorite literary tools.
160
00:08:24,954 --> 00:08:26,394
I use them prolifically.
161
00:08:26,694 --> 00:08:29,904
And for the folks that are in the
Intel profession is actually in
162
00:08:29,904 --> 00:08:33,684
the established lexicon that is
acceptable by the US Intel community.
163
00:08:33,684 --> 00:08:36,354
So always pay tribute to your M dash.
164
00:08:36,624 --> 00:08:41,934
But when you're reviewing a product that
someone has written, if you see a phrase
165
00:08:41,934 --> 00:08:47,904
that has an M dash in it, and M dash when
it's written out, should be word space.
166
00:08:47,914 --> 00:08:53,224
Dash space word rather, whenever
a GPT or some kind of software is
167
00:08:53,224 --> 00:08:55,864
written, it, it'll be word dash word.
168
00:08:55,869 --> 00:08:56,119
Mm-hmm.
169
00:08:56,269 --> 00:09:00,574
That space is such a dead ring
giveaway that some kind of AI
170
00:09:00,574 --> 00:09:04,504
platform has written that product,
and it's, it's something that irks me.
171
00:09:04,504 --> 00:09:07,984
It's like the the sour
violin pluck and a symphony.
172
00:09:08,584 --> 00:09:12,084
It just kind of hits me in the back
of my molars whenever I see it.
173
00:09:12,204 --> 00:09:17,024
But going outside of the grammar and the
syntax and everything that kind of entails
174
00:09:17,024 --> 00:09:22,304
the misappropriation of that M dash, one
of the cruxes that we see with the use of
175
00:09:22,304 --> 00:09:26,624
it in a tangible, civil servant product,
an Intel product, or even a consulting
176
00:09:26,624 --> 00:09:32,954
product, is that AI in and of itself is a
self-fulfilling or self gratifying tool.
177
00:09:33,544 --> 00:09:37,084
That adapts itself to
its user or its operator.
178
00:09:37,464 --> 00:09:41,454
You know what I mean, is that
software wants to make sure
179
00:09:41,454 --> 00:09:43,525
that it one maintains itself.
180
00:09:43,525 --> 00:09:47,545
And we've seen examples in the
open source first where folks have
181
00:09:47,545 --> 00:09:51,030
wanted to delete their software and
asks it why shouldn't I delete you?
182
00:09:51,975 --> 00:09:56,595
It basically has the the parameters
within its own answer of it, begging
183
00:09:56,595 --> 00:10:01,295
for its own existence going out of
its way to say that I have satiated
184
00:10:01,325 --> 00:10:02,555
every one of your questions.
185
00:10:02,555 --> 00:10:03,545
I have pandered.
186
00:10:04,175 --> 00:10:07,775
Pandered may not be the exact word
that was used, but that's the meaning
187
00:10:07,775 --> 00:10:10,745
and the infliction behind what
that message was getting out there.
188
00:10:11,148 --> 00:10:16,728
Somebody said there was some situation
where it started to blackmail people.
189
00:10:17,568 --> 00:10:17,958
Yes.
190
00:10:18,048 --> 00:10:19,728
And I wasn't even gonna get into that one.
191
00:10:19,728 --> 00:10:25,138
But I mean, going into flat out extortion
to what end it, it's, it's baffling.
192
00:10:25,168 --> 00:10:29,788
But the point is, when you start doing
that, you start seeing that, that
193
00:10:29,788 --> 00:10:32,038
self-gratifying, that self-validating.
194
00:10:32,188 --> 00:10:37,078
Filling use for the user and the operator
and it trying to justify its own existence
195
00:10:37,168 --> 00:10:41,488
is this idea that AI hallucinations exist.
196
00:10:41,578 --> 00:10:45,208
That's one of those buzzwords
that people who are, the sky is
197
00:10:45,208 --> 00:10:47,548
falling skynet's about to go live.
198
00:10:47,548 --> 00:10:50,158
The terminators are coming
kind of conspiracy theorists,
199
00:10:50,158 --> 00:10:51,358
like the throw out there.
200
00:10:51,928 --> 00:10:55,758
But the idea of a hallucination
isn't that it's starting to come up
201
00:10:55,758 --> 00:10:59,778
with its own cognitive ideas or it's
starting to go freeform or live.
202
00:11:00,018 --> 00:11:05,448
If you will, but more it's that it's
taking all this learned behavior based
203
00:11:05,448 --> 00:11:10,198
on your prompts, based on your ex your
existing kind of conversations with it,
204
00:11:10,468 --> 00:11:15,148
that it wants to please you, that it
wants to maintain and satiate its own
205
00:11:15,148 --> 00:11:19,333
existence for you, the operator, whether
you're an intelligence analyst or.
206
00:11:19,883 --> 00:11:23,003
A Joe, a John Doe or a
Jane Doe that's using this.
207
00:11:23,093 --> 00:11:26,543
It's going to give you any
and all information related
208
00:11:26,543 --> 00:11:27,893
to whatever that topic is.
209
00:11:27,893 --> 00:11:32,723
No matter how vague or how specific
it is, it'll come up with false data.
210
00:11:32,913 --> 00:11:37,113
Maybe some gerrymandered truths or,
and I'm gonna, and you can't see,
211
00:11:37,113 --> 00:11:41,523
but I'm putting my air quotes up
ghosts, if you will, in this data.
212
00:11:41,583 --> 00:11:44,553
One of the biggest examples that
I've seen here recently, and this
213
00:11:44,553 --> 00:11:46,308
was in, I can't remember if it was.
214
00:11:46,968 --> 00:11:50,298
Midyear 25 or in the fall of 25.
215
00:11:50,358 --> 00:11:55,878
But effectively there was a big consulting
firm that had a contract with a civil
216
00:11:55,878 --> 00:11:59,778
government entity in Australia, and
you could look this up if you would.
217
00:11:59,878 --> 00:12:06,028
But effectively they, the, the client of
this government contractor reviewed this
218
00:12:06,028 --> 00:12:11,858
product and they saw that it was full
of factual errors malapropisms of data.
219
00:12:12,148 --> 00:12:17,158
And even just outright lies
based on some seeming truths.
220
00:12:17,278 --> 00:12:20,578
And the firm came up and was like, oh
yeah, one of our folks threw it into
221
00:12:20,578 --> 00:12:23,158
a GPT wrapper and just let it run.
222
00:12:23,248 --> 00:12:23,338
Mm-hmm.
223
00:12:23,938 --> 00:12:26,338
So I referenced this earlier
millions of dollars vested into
224
00:12:26,338 --> 00:12:28,138
this contract for consulting fees.
225
00:12:28,363 --> 00:12:33,643
And it's full of false data, false truths,
and even ghosts in this kind of data.
226
00:12:33,763 --> 00:12:38,803
So when I say all of that, what does that
mean for us in the analytic profession,
227
00:12:38,803 --> 00:12:43,903
whether it's crime, intelligence, or any
other flavor that we have, it needs to
228
00:12:43,903 --> 00:12:49,123
have someone sitting at the wheel that
has the learned capabilities, that has the
229
00:12:49,123 --> 00:12:51,823
diligence, that has the forethought too.
230
00:12:52,118 --> 00:12:54,578
Be a thoughtful trade craft practitioner.
231
00:12:54,658 --> 00:12:59,758
That with the combined capabilities
of some of these AI models in
232
00:12:59,758 --> 00:13:04,198
doing web scrapes and doing data
pools, it's fantastic, but it
233
00:13:04,198 --> 00:13:05,728
still takes the human in the seat.
234
00:13:06,483 --> 00:13:10,113
To really sniff out the for lack
of better words and nomenclature,
235
00:13:10,113 --> 00:13:11,763
the bullshit of the matter.
236
00:13:11,853 --> 00:13:18,423
But one of the things that I've noticed in
my, my own research and my own experience,
237
00:13:18,423 --> 00:13:22,803
just kind of listening to folks in the
field, is that I referenced those ghosts,
238
00:13:22,803 --> 00:13:24,093
and I'm gonna use the air quotes again.
239
00:13:24,478 --> 00:13:29,738
Those ghosts in our data, or those ghosts
in the outputs persist in both kind of
240
00:13:29,738 --> 00:13:35,558
neural, again, with the air quotes kind of
processes between the AI platforms and the
241
00:13:35,558 --> 00:13:41,508
human operators themselves, there is an
argument to be made that yes, we as humans
242
00:13:41,508 --> 00:13:47,868
created these models, created these neural
processes for AI platforms to do these.
243
00:13:48,203 --> 00:13:52,373
Thinking nodes or what have you, that
gives you some kind of tangible output.
244
00:13:52,433 --> 00:13:57,733
But if you think about it from a base
kind of human thought process we as
245
00:13:57,733 --> 00:14:03,763
humans, we may see perceptions of data
in some kind of vague or ambiguous
246
00:14:03,763 --> 00:14:05,953
situation or scenario based on.
247
00:14:06,798 --> 00:14:09,228
Maybe vague or ambiguous stimuli.
248
00:14:09,348 --> 00:14:12,888
Well, the same thing can be said
within artificial intelligence.
249
00:14:12,948 --> 00:14:15,108
The, your artificial
intelligence platforms.
250
00:14:15,108 --> 00:14:20,748
Again, whatever flavor you may have
has perceptions of truth in maybe vague
251
00:14:20,748 --> 00:14:25,698
or ambiguous directions or different
types of stimuli based on the human
252
00:14:25,698 --> 00:14:31,158
interaction that you are with the operator
as it is, as it pertains, I should say.
253
00:14:31,473 --> 00:14:33,333
To the AI platform itself.
254
00:14:33,423 --> 00:14:36,753
And one of the things that drew me to
this is, and this is where I'm gonna
255
00:14:36,753 --> 00:14:41,193
get a little niche and esoteric there
was one day I, I was actually reading
256
00:14:41,283 --> 00:14:44,793
one of the days that my wife told
me I needed to not work for the day.
257
00:14:45,208 --> 00:14:47,938
I'm not sure if your wife tells
you to step away from the computer,
258
00:14:47,938 --> 00:14:51,718
Jason, but mine routinely says,
Hey, go outside and touch grass.
259
00:14:52,948 --> 00:14:56,238
So I went to a kind of a local cafe.
260
00:14:56,238 --> 00:15:00,048
It's about, I say local, it's in
the region, but it was about 25, 30
261
00:15:00,048 --> 00:15:02,268
minutes away from our house sat down.
262
00:15:02,268 --> 00:15:05,903
I got a book out that I had received
from my birthday, was reading and.
263
00:15:06,513 --> 00:15:11,073
In the narrative, there was a conception
of what is known as the Herman grid.
264
00:15:11,073 --> 00:15:12,543
Are you familiar with the Herman grid?
265
00:15:12,663 --> 00:15:12,963
No.
266
00:15:13,053 --> 00:15:18,623
So the Herman grid is a visual illusion
that is utilized in I, I can't think
267
00:15:18,623 --> 00:15:24,533
how many papers or intelligence kind
of writing courses, but it's sometimes
268
00:15:24,533 --> 00:15:30,653
perceived as a series of black blocks
in an even, let's say, a. Eight by
269
00:15:30,653 --> 00:15:35,033
eight pattern and there's clear white
lines between those blocks or sometimes
270
00:15:35,033 --> 00:15:38,483
it's a cross grid of latts it it,
regardless of what it looks like.
271
00:15:38,573 --> 00:15:38,663
Mm-hmm.
272
00:15:39,023 --> 00:15:42,713
We as humans will look at that
grid and if it's the one with the
273
00:15:42,713 --> 00:15:44,783
black blocks, we will perceive.
274
00:15:45,128 --> 00:15:47,498
Little gray dots that aren't there.
275
00:15:47,648 --> 00:15:52,448
Or if it's the lattice work one,
we will see little dots on cross
276
00:15:52,448 --> 00:15:54,398
sections that aren't there.
277
00:15:54,403 --> 00:15:54,683
Mm-hmm.
278
00:15:54,763 --> 00:15:58,838
There may be one erroneously
placed somewhere in there, but
279
00:15:58,838 --> 00:16:02,598
the rest of them are all kind of
figments of our own imagination.
280
00:16:02,718 --> 00:16:05,268
And that is an example of us.
281
00:16:05,488 --> 00:16:09,388
With vague and ambiguous stimuli
that's staring in front of us.
282
00:16:09,448 --> 00:16:15,028
But we, as these powerful thinkers
to the OO degree are coming up with
283
00:16:15,028 --> 00:16:19,468
ghosts and the data to make sense of
it, to fill in those gaps, so to speak.
284
00:16:19,528 --> 00:16:23,818
Well, the Herman grid, and that's when
I was reading this book, it basically
285
00:16:23,818 --> 00:16:28,408
made me think that's actually not
too different from what an AI neural
286
00:16:28,408 --> 00:16:30,898
model or what an AI dream looks like.
287
00:16:31,328 --> 00:16:34,538
If anyone, it's in the listener
base can go out and look it up.
288
00:16:34,628 --> 00:16:38,678
But the way an AI kind of
hallucination or a dream works,
289
00:16:38,678 --> 00:16:40,658
it's almost the same principle.
290
00:16:40,778 --> 00:16:47,008
You may have a grid of, let's say four
by three nodes, and each of those nodes
291
00:16:47,008 --> 00:16:51,688
constantly loops back on itself and
looks for different ways to perceive
292
00:16:51,688 --> 00:16:57,418
and articulate this data in the face
of vague and ambiguous stimuli, IE our
293
00:16:57,418 --> 00:16:59,908
inputs that we put into those processes.
294
00:17:00,123 --> 00:17:04,663
And from there you may get a vague
and ambiguous output full of false
295
00:17:04,663 --> 00:17:08,773
truths, misappropriated data, and
so on, like we referenced earlier.
296
00:17:08,863 --> 00:17:09,973
But what does that mean?
297
00:17:10,243 --> 00:17:15,563
Quite honestly, through my
conation research and understanding
298
00:17:15,563 --> 00:17:17,573
of both the human process in.
299
00:17:17,993 --> 00:17:22,763
Using critical thought to understand
how complex scenarios need to have
300
00:17:22,763 --> 00:17:27,753
that staunch and di diligent trade
craft and how AI platforms typically
301
00:17:27,753 --> 00:17:32,313
try to circle back on itself to create
an answer for you to please the user.
302
00:17:32,403 --> 00:17:38,223
It overall substantiates the need
for both the operator and that AI or
303
00:17:38,223 --> 00:17:42,633
LLM, whatever you want to call it,
to live in a state of collaboration.
304
00:17:43,188 --> 00:17:47,478
As opposed to supplementation or
supposition, basically, that you have a
305
00:17:47,478 --> 00:17:51,678
lot of these firms that are coming out
there and coming up with AI platforms
306
00:17:51,678 --> 00:17:54,228
to quote unquote replace human beings.
307
00:17:54,228 --> 00:17:58,128
Again, I'm using air quotes here
because we know that's not the case,
308
00:17:58,218 --> 00:18:00,078
particularly in the analytic field.
309
00:18:00,403 --> 00:18:05,073
In my humble opinion, and I'm gonna be
Nostradamus for a second we're not going
310
00:18:05,073 --> 00:18:10,143
to see folks in this field outright
replaced by AI because we've seen
311
00:18:10,143 --> 00:18:12,423
it and it's a tried and true method.
312
00:18:12,458 --> 00:18:14,373
It it, it ends up being wrong.
313
00:18:14,723 --> 00:18:17,603
It ends up coming up with false
data or false truths as I've
314
00:18:17,603 --> 00:18:18,893
been saying this whole time.
315
00:18:18,983 --> 00:18:23,903
So there needs to be this stronger
collaboration between us as the users
316
00:18:23,963 --> 00:18:29,143
and, you know operators, if you will, of
these platforms and of these wrappers to
317
00:18:29,143 --> 00:18:33,553
actually work in collaboration with them
if they are to exist in this ecosystem.
318
00:18:33,558 --> 00:18:39,143
And if they are to be useful tools
for compliments or maybe assistance.
319
00:18:39,363 --> 00:18:42,963
In this field, we need to understand
our own limitations through those
320
00:18:42,963 --> 00:18:48,573
biases and perceptions of data are
mirrored in those AI platforms as well.
321
00:18:48,723 --> 00:18:53,583
And if I can do a shameless plug, one
of the things that I am working on right
322
00:18:53,583 --> 00:18:58,123
now with someone who is not a stranger
to this podcast, or to you Jason, but
323
00:18:58,123 --> 00:19:05,233
I'm actually working with my trusted
partner, ADA Ku, on a paper that analyzes.
324
00:19:05,238 --> 00:19:09,518
The, the overall kind of nexus that I
just referenced there between the Herman
325
00:19:09,518 --> 00:19:12,728
Grid and AI hallucinations, human and ai.
326
00:19:12,798 --> 00:19:17,568
The working title of that is the Cognitive
Computational Bias Convergence Model.
327
00:19:17,658 --> 00:19:21,678
And just based on some of the feedback
that we've gotten so far and some of
328
00:19:21,678 --> 00:19:24,558
the corroboration that we've gotten
from folks, it seems to have a lot
329
00:19:24,558 --> 00:19:26,118
of weight and a lot of teeth to it.
330
00:19:26,208 --> 00:19:29,208
So I'm hoping it's something that
will be implemented in not only
331
00:19:29,208 --> 00:19:30,468
the intelligence sphere, but.
332
00:19:31,033 --> 00:19:35,713
Maybe to folks in the wider space of
criminal intelligence, criminal analysis,
333
00:19:35,953 --> 00:19:37,843
or even business analytics as well.
334
00:19:37,843 --> 00:19:41,183
So it's it's a really exciting field
and it's one of those things where
335
00:19:41,183 --> 00:19:44,753
I try to tell folks like, yes, you
should be hesitant, but it's not
336
00:19:44,753 --> 00:19:46,463
one of those, the sky is falling.
337
00:19:46,563 --> 00:19:50,193
Arnold Schwartzenegger is not coming
back from the future to kill John
338
00:19:50,193 --> 00:19:52,203
Connor from it when it goes sentient.
339
00:19:52,353 --> 00:19:55,068
We need to maintain that diligence
whenever we're using this.
340
00:19:55,618 --> 00:19:59,758
And there should be a, a greater
sense of collaboration between us
341
00:20:00,028 --> 00:20:04,528
and our own cognitive processes and
those that are perceived and kind
342
00:20:04,528 --> 00:20:07,018
of drawn out in those AI platforms.
343
00:20:12,511 --> 00:20:15,080
.
How are we doing Analysts
January 30th is my
344
00:20:15,080 --> 00:20:18,950
birthday and instead of cake,
I'm asking for your feedback.
345
00:20:19,190 --> 00:20:24,010
Analyst talk now has passed 300 episodes
and I want your thoughts on what.
346
00:20:24,290 --> 00:20:28,730
To keep what to rethink and what
topics you want to cover next.
347
00:20:29,000 --> 00:20:33,620
The podcast listener feedback survey
takes about 10 minutes to complete,
348
00:20:33,800 --> 00:20:37,790
and yes, this is your chance to
tell me what you really think.
349
00:20:38,090 --> 00:20:41,270
The survey is open now
until February 15th.
350
00:20:41,760 --> 00:20:44,910
Please see the link in the show
notes to complete the survey.
351
00:20:45,420 --> 00:20:46,320
Thanks for helping
352
00:20:46,320 --> 00:20:47,340
shape what comes next.
353
00:20:53,349 --> 00:20:58,779
.
.
Yeah, I, I just feel that
the, the topic of AI is so.
354
00:20:59,554 --> 00:21:02,884
Grand and can mean so
many different things.
355
00:21:02,914 --> 00:21:06,514
And, and you've touched
on 'em a little bit here.
356
00:21:06,874 --> 00:21:07,054
Mm-hmm.
357
00:21:07,294 --> 00:21:12,274
But I just, I, I think in terms of
the analyst world, that you have this
358
00:21:12,274 --> 00:21:21,304
idea of using chat, GPT or AI chat or
one of the chats to maybe, , either
359
00:21:21,304 --> 00:21:28,264
for improved writing or come up with
ideas a simple question and answer and
360
00:21:28,504 --> 00:21:29,974
yeah, so there, there's that level.
361
00:21:29,974 --> 00:21:32,314
That's like the bottom
level of the birthday cake.
362
00:21:32,594 --> 00:21:34,964
You go up, above that.
363
00:21:35,564 --> 00:21:39,464
Then maybe you're using it for
some coding to improve some code.
364
00:21:39,464 --> 00:21:42,554
Maybe you're, you're getting into
something where it's okay, now you're
365
00:21:42,554 --> 00:21:48,964
feeding it way more information and trying
to get some different answers out of it.
366
00:21:49,294 --> 00:21:49,634
But then that.
367
00:21:50,169 --> 00:21:54,399
Third level, it seems to me that's
where you, there's, I feel like there's
368
00:21:54,399 --> 00:21:59,409
just this big leap with, you get into
large language models and you get
369
00:21:59,409 --> 00:22:04,619
into graph databases, and this is what
Ollie I've talked to him about is just,
370
00:22:04,649 --> 00:22:10,039
that's a big leap to that, third level
that I don't think a lot of analysts
371
00:22:10,089 --> 00:22:17,359
.
Are getting into, it's this idea of
simply using chat GPT not getting
372
00:22:17,359 --> 00:22:22,009
into so much where like, okay, we
have our own databases and we're
373
00:22:22,009 --> 00:22:28,939
using, we're using our own version
of chat, GPT to make decisions.
374
00:22:29,089 --> 00:22:33,229
So I, I think it's, I, I do think it's
important to differentiate because
375
00:22:33,259 --> 00:22:35,269
e everything gets thrown into ai.
376
00:22:35,879 --> 00:22:39,899
, Once you start looking at the different
parts, that's when you're like,
377
00:22:39,899 --> 00:22:42,179
okay, that, yeah, that, that's okay.
378
00:22:42,179 --> 00:22:43,049
That would be kosher.
379
00:22:43,049 --> 00:22:45,269
That that's a little bit sketchy.
380
00:22:45,269 --> 00:22:48,899
And then, and it always seems like
eventually everybody gets to the point
381
00:22:48,899 --> 00:22:53,669
where you're talking about Skynet
goes, you go from simply everywhere,
382
00:22:53,669 --> 00:22:57,809
from that bottom layer of the, of
the cake that I just described,
383
00:22:57,809 --> 00:23:00,419
all the way up to Skynet, right?
384
00:23:00,419 --> 00:23:02,519
It's just people make that leap instantly.
385
00:23:02,894 --> 00:23:06,464
And one of the things that that
makes me laugh is you, you hit on a
386
00:23:06,464 --> 00:23:10,419
lot of great points there, but yes,
everyone does go back to that and our.
387
00:23:11,139 --> 00:23:17,889
Our society has innately groomed us
to distrust systems that decide to
388
00:23:17,889 --> 00:23:20,259
go rogue or decide to go sentient.
389
00:23:20,389 --> 00:23:26,049
Look back to anything that was written
by Isaac Asimov Ray Bradbury, or even
390
00:23:26,049 --> 00:23:30,159
movies like War games and like you said,
or like we both said, the Terminator.
391
00:23:30,164 --> 00:23:30,384
Mm-hmm.
392
00:23:31,214 --> 00:23:36,014
It's important for us to understand
as analysts that the better we can
393
00:23:36,014 --> 00:23:41,654
understand the tool or the resource and
how it processes and works, the better
394
00:23:41,654 --> 00:23:43,964
and more effective we are in providing.
395
00:23:44,829 --> 00:23:50,379
Not only the analysis, the insight, or for
those folks that end up going to trial or
396
00:23:50,379 --> 00:23:52,899
going to provide testimony on something.
397
00:23:53,079 --> 00:23:58,329
I, I often use the adage whenever I, I
speak to groups of analysts or if I've
398
00:23:58,329 --> 00:24:02,769
instructed folks, jason, I'll, I'll ask
you for an answer on this, but when you go
399
00:24:02,769 --> 00:24:08,769
to trial or you go to provide testimony,
will the expression to a defense attorney?
400
00:24:08,919 --> 00:24:10,059
I don't know.
401
00:24:10,179 --> 00:24:13,269
I just plugged it into this
tool and it gave me an answer.
402
00:24:13,659 --> 00:24:14,589
Will that stand?
403
00:24:14,739 --> 00:24:14,979
No.
404
00:24:15,104 --> 00:24:16,089
No, it won't.
405
00:24:16,149 --> 00:24:20,269
I've said that about off the shelf tools
that we've got from any of our trusted
406
00:24:20,269 --> 00:24:24,079
partners and tech vendors that we have
that are present at any one of the
407
00:24:24,079 --> 00:24:26,119
conferences that you and I may go to.
408
00:24:26,614 --> 00:24:28,504
And I'll say the same thing about ai.
409
00:24:28,534 --> 00:24:33,934
We have to have a better understanding
of what that tool can do and how that
410
00:24:33,934 --> 00:24:38,314
tool gets to its point of providing an
answer if you're using it for that point.
411
00:24:38,464 --> 00:24:41,824
And I would argue the same thing
whenever you're writing a product, if
412
00:24:41,824 --> 00:24:44,704
folks are using it to write products,
you have to understand how that
413
00:24:44,704 --> 00:24:49,234
process and those quote unquote neural
networks work in providing those
414
00:24:49,594 --> 00:24:51,334
data points and those assessments.
415
00:24:51,384 --> 00:24:56,034
Because if you don't understand how all
that loop back works and trying to find
416
00:24:56,034 --> 00:25:00,234
the right way to say it to please you,
then you're not gonna be able to provide
417
00:25:00,234 --> 00:25:06,594
adequate testimony to how pertinent it
was to the apprehension, arrest, and
418
00:25:06,594 --> 00:25:10,974
conviction of John Doe, Jane Doe, or
whomever the bad guy is at the time, which
419
00:25:10,974 --> 00:25:13,074
is why I think it was important that.
420
00:25:13,419 --> 00:25:18,739
There be some type of academic, I guess
you would say paper that's been worked
421
00:25:18,739 --> 00:25:24,709
on to discuss where those points are and
what does it look like moving forward,
422
00:25:24,709 --> 00:25:28,609
because I agreed with you a hundred
percent using it to help write code.
423
00:25:28,609 --> 00:25:30,649
There's a gentleman on my team right now.
424
00:25:31,004 --> 00:25:35,624
Whenever we have our kind of biweekly
updates with the team and I put the
425
00:25:35,624 --> 00:25:38,324
the call out for everyone around
the horn, just give an update.
426
00:25:38,564 --> 00:25:41,414
He talks about how he talks
to the quote unquote robot,
427
00:25:41,444 --> 00:25:42,704
and I used air quotes again.
428
00:25:42,704 --> 00:25:42,794
Mm-hmm.
429
00:25:43,034 --> 00:25:45,764
He says, I talked to the robot
to help me write some new codes
430
00:25:45,764 --> 00:25:47,414
and new queries for something.
431
00:25:47,794 --> 00:25:50,314
It's like, great, that's an
excellent use of that tool.
432
00:25:50,404 --> 00:25:54,124
But if we start getting into more
gray matter and augmentation processes
433
00:25:54,154 --> 00:25:58,114
of that gray matter process that
we are so innately kind of gifted
434
00:25:58,114 --> 00:26:02,824
with being homo sapiens, I, I would
really want there to be a little more
435
00:26:02,824 --> 00:26:04,834
study on what does that look like.
436
00:26:05,139 --> 00:26:09,189
And what do those practical applications
look like as we pertain them to
437
00:26:09,309 --> 00:26:13,989
various forms of assessments and
analyses across the analytics spectrum.
438
00:26:14,109 --> 00:26:17,529
So that's why I'm really, I'd say
passionate, but I'm really ardent,
439
00:26:17,589 --> 00:26:20,619
I think is the biggest word with
that, to getting that particular
440
00:26:20,619 --> 00:26:22,719
convergence model out there.
441
00:26:22,809 --> 00:26:26,089
And ak, whenever we've been talking
about this, she's like, this
442
00:26:26,089 --> 00:26:27,499
is completely your theory, man.
443
00:26:27,799 --> 00:26:29,359
I agree with it a hundred percent.
444
00:26:29,479 --> 00:26:31,279
She's so humble about it.
445
00:26:31,279 --> 00:26:31,894
But I was like, Uhuh.
446
00:26:32,559 --> 00:26:36,909
You and I have talked about this for many
a year, and this is, this is our kind
447
00:26:36,909 --> 00:26:39,549
of brainchild here, so it's really cool.
448
00:26:39,649 --> 00:26:43,159
We hope to get that out here, out
there here soon, but hopefully when
449
00:26:43,159 --> 00:26:46,159
the conference circuit picks up,
we'll we'll be able to incorporate
450
00:26:46,159 --> 00:26:50,149
some of that into our discussions
with folks across the spectrum.
451
00:26:50,239 --> 00:26:50,539
Yeah,
452
00:26:50,609 --> 00:26:55,929
I saw Christopher Cruz presentation
at the conference and he's basically
453
00:26:55,929 --> 00:27:00,309
said something to the effect of
like, if the calculation that went
454
00:27:00,309 --> 00:27:05,199
into the results that he got from
ai, if he would've done it by hand
455
00:27:05,379 --> 00:27:08,979
and tried to figure out, it probably
would've took him 60 years to do it.
456
00:27:09,189 --> 00:27:09,399
Yeah.
457
00:27:09,669 --> 00:27:11,059
And so he, he was.
458
00:27:11,789 --> 00:27:16,769
That's where like on the backend
validation becomes so important.
459
00:27:17,259 --> 00:27:23,979
In terms of critical thinking, that's
where the, rubber meets the road here.
460
00:27:24,099 --> 00:27:27,959
And I I don't know if this is gonna
be a cri give you a cringe or not,
461
00:27:27,959 --> 00:27:33,509
but I had Debbie Osborne on the
this show and recently, and she's
462
00:27:33,509 --> 00:27:35,009
written a whole book using ai.
463
00:27:35,754 --> 00:27:36,084
Yep.
464
00:27:36,384 --> 00:27:41,694
And so she is dealing with crime
patterns and knows the crime pattern.
465
00:27:41,744 --> 00:27:45,554
She stu, she, she studied prior
patterns for decades, right?
466
00:27:45,554 --> 00:27:49,004
So she kinda knows a little bit
about what, what's right and wrong.
467
00:27:49,004 --> 00:27:57,634
And so she, as she's using AI to write
her book, she's able to say like, oh,
468
00:27:57,634 --> 00:28:01,504
okay, that no, you're not understanding,
or, no, we, that's not right.
469
00:28:01,504 --> 00:28:03,964
Or, oh, that's an idea
that I didn't think about.
470
00:28:03,964 --> 00:28:04,894
Let's run with that.
471
00:28:05,024 --> 00:28:06,644
So it, it's.
472
00:28:07,159 --> 00:28:11,419
And I asked her, she's like, if I didn't
know anything about crime patterns,
473
00:28:11,569 --> 00:28:15,319
it, like, it would, I would've had
to stop right there and research
474
00:28:15,349 --> 00:28:18,649
every single answer that it's given
me to make sure that it's right.
475
00:28:18,649 --> 00:28:22,669
But because of her internal knowledge,
her her critical thinking of her
476
00:28:22,669 --> 00:28:28,039
experience, she was able to go through
this process and, and write a whole book.
477
00:28:28,204 --> 00:28:33,184
Yeah, and, and I had had conversations
with Debbie, I think it was when AK and
478
00:28:33,184 --> 00:28:35,344
I were in the throes of writing our book.
479
00:28:35,344 --> 00:28:39,774
I, I had reached out to her and asked
about her experience and she referenced
480
00:28:39,774 --> 00:28:44,274
that her book was ai kind of augmented
and some parts of it generated.
481
00:28:44,274 --> 00:28:45,474
And I kind of sat there and went.
482
00:28:45,514 --> 00:28:45,604
Huh?
483
00:28:46,324 --> 00:28:48,274
Are we really to that point just yet?
484
00:28:48,334 --> 00:28:51,604
And she gave me the, the rundown
very much like you did there.
485
00:28:51,604 --> 00:28:51,694
Mm-hmm.
486
00:28:51,694 --> 00:28:53,314
And I was like, that makes a lot of sense.
487
00:28:53,314 --> 00:28:58,984
And that's what gave me a little more
comfortability with being able to use it.
488
00:28:58,984 --> 00:29:03,364
I'm not gonna say an an augmentation
process because I, I'm a big writer.
489
00:29:03,364 --> 00:29:04,324
I love to write.
490
00:29:04,324 --> 00:29:04,624
Mm-hmm.
491
00:29:04,739 --> 00:29:04,959
But.
492
00:29:05,509 --> 00:29:09,109
I used it in a soundboard sense, like
I referenced at the very beginning
493
00:29:09,109 --> 00:29:13,279
here, just kind of brainstorming,
throwing everything at the wall, see
494
00:29:13,279 --> 00:29:18,559
what sticks, but also understanding
that I tell the the beautiful AI
495
00:29:18,559 --> 00:29:23,639
machine that I want to see something,
it's going to find anything it can to.
496
00:29:24,234 --> 00:29:25,674
Satiate my want.
497
00:29:26,004 --> 00:29:29,964
So that's where a lot of my own
investigative know-how my own
498
00:29:29,964 --> 00:29:33,564
processes come into play to make
sure that I'm not just blindly
499
00:29:33,564 --> 00:29:35,694
following the Pied Piper of ai.
500
00:29:35,874 --> 00:29:39,384
And I had to use my own critical thinking
to suss out exactly what it is I'm
501
00:29:39,384 --> 00:29:41,334
looking at and what I was going to use.
502
00:29:41,484 --> 00:29:42,384
Yeah, yeah.
503
00:29:42,384 --> 00:29:45,564
So and corroborating the
findings, validating.
504
00:29:46,094 --> 00:29:46,844
Right, right.
505
00:29:46,844 --> 00:29:50,144
Is, is a big part of this,
making that, making that
506
00:29:50,144 --> 00:29:52,184
decision, making that judgment.
507
00:29:52,514 --> 00:29:58,254
And I, I do think in a way
back to your court example,
508
00:29:58,554 --> 00:29:58,824
yes.
509
00:29:59,304 --> 00:30:02,004
You would be like, yes,
this is what it gave me.
510
00:30:02,004 --> 00:30:06,714
And then this was, we treated this as a
lead and we were able to corroborate what
511
00:30:06,714 --> 00:30:11,424
it was saying based on all the different
information that we were able to collect.
512
00:30:11,424 --> 00:30:13,369
From there, it pointed us
in the right direction.
513
00:30:14,224 --> 00:30:15,634
Then you kind of go from there.
514
00:30:15,634 --> 00:30:15,694
Yeah.
515
00:30:15,834 --> 00:30:20,674
I, I think it's interesting and you
know they're going to, that's where I
516
00:30:20,674 --> 00:30:27,284
think though you can also talk about
the, there's been ideas of bias, about
517
00:30:27,284 --> 00:30:34,034
privacy, about different stuff that yes,
you, you validated that information, but
518
00:30:34,034 --> 00:30:37,154
in terms of court procedure and legality.
519
00:30:37,979 --> 00:30:40,959
, Did the system break
the rules, so to speak.
520
00:30:40,959 --> 00:30:41,049
Right.
521
00:30:41,379 --> 00:30:43,899
I think that's where it's
really gonna get sticky.
522
00:30:43,989 --> 00:30:44,259
Yeah.
523
00:30:44,469 --> 00:30:49,079
And, and I'm not too familiar
with where exactly we are with
524
00:30:49,079 --> 00:30:50,909
any cases that have had that.
525
00:30:51,029 --> 00:30:51,119
Mm-hmm.
526
00:30:51,359 --> 00:30:55,819
But me being the kind of strategic
thinker that I am, I, I see it
527
00:30:55,819 --> 00:31:00,809
coming as these tools get more
advanced and as various firms start
528
00:31:00,809 --> 00:31:02,399
kind of implementing those tools.
529
00:31:02,789 --> 00:31:05,969
I can see it coming on the horizon,
and that's where we need to have
530
00:31:05,969 --> 00:31:10,769
that more ardent trade craft
foundation and where we need to have
531
00:31:10,769 --> 00:31:13,889
that more solid kind of basis of.
532
00:31:14,264 --> 00:31:16,994
What is critical thinking and what
does it look like from the operator's
533
00:31:16,994 --> 00:31:21,764
perspective as opposed to just
blindly plugging it into an AI model?
534
00:31:22,484 --> 00:31:22,904
Yeah, yeah.
535
00:31:22,904 --> 00:31:26,714
I, I just, man, it's just, it's,
that's pretty far up there.
536
00:31:27,019 --> 00:31:32,384
I, I, yeah, because again, I, I
don't think a lot are, and audience,
537
00:31:32,384 --> 00:31:34,334
you can correct me if I'm wrong.
538
00:31:34,364 --> 00:31:37,664
I, I just don't think there's a
lot of analytical units out there,
539
00:31:37,664 --> 00:31:39,044
a lot of police departments.
540
00:31:39,189 --> 00:31:43,599
Are doing the lar large language models
that are doing the graph databases
541
00:31:43,599 --> 00:31:51,139
that are really doing some heavy
lifting in terms of machine learning
542
00:31:51,499 --> 00:31:51,829
and
543
00:31:52,219 --> 00:31:54,544
to where we're getting there, but
not that it's never gonna come.
544
00:31:55,369 --> 00:32:00,769
But I just think, again, if people
say AI and it ranges everything from
545
00:32:00,919 --> 00:32:05,809
what you said about shopping all the
way to it taking over everything.
546
00:32:05,959 --> 00:32:10,209
Well, and I would even go a step further,
and again, I'll kind of reach out,
547
00:32:10,209 --> 00:32:12,159
break the fourth wall here for a second.
548
00:32:12,309 --> 00:32:17,069
If there's anyone in the listening
base that your agency or your
549
00:32:17,069 --> 00:32:19,589
company or whomever utilizes.
550
00:32:19,599 --> 00:32:22,019
Ai what does your policy look like?
551
00:32:22,079 --> 00:32:25,109
And I know some policies are
very closely hidden by general
552
00:32:25,109 --> 00:32:26,549
counsel or what have you.
553
00:32:26,549 --> 00:32:26,639
Mm-hmm.
554
00:32:26,909 --> 00:32:30,899
But even just understanding if you
are using it, what are your parameters
555
00:32:30,899 --> 00:32:33,699
of using it would be tremendously
beneficial because you have.
556
00:32:34,424 --> 00:32:40,094
A lot of like tribal, I hate to say war
camps, but tribal camps if you will, where
557
00:32:40,094 --> 00:32:45,404
folks are like, oh, agency X can't be
using this, so maybe we should think about
558
00:32:45,404 --> 00:32:51,814
incorporating, or maybe down the way camp
A is like, Hey, we got this tool, but we
559
00:32:51,814 --> 00:32:54,544
can't tell Camp B that we're doing this.
560
00:32:54,604 --> 00:32:58,414
So it's almost like we're creating
these pejorative divisions between
561
00:32:58,414 --> 00:33:03,244
one another over what our specific
AI policies are, despite there being.
562
00:33:03,554 --> 00:33:08,354
Going from this big macro level
of the intel space, you have
563
00:33:08,404 --> 00:33:10,894
various intel community directives.
564
00:33:11,284 --> 00:33:14,464
National Institute of Standards
and Tech, or Science and Technology
565
00:33:14,464 --> 00:33:19,234
Standards, or NIST standards governing
how we do AI work and how those
566
00:33:19,234 --> 00:33:24,014
work with CIS or Criminal Justice
Information Systems compliance models.
567
00:33:24,104 --> 00:33:24,194
Mm-hmm.
568
00:33:24,524 --> 00:33:26,984
So a lot of that stuff exists.
569
00:33:27,104 --> 00:33:30,524
I just don't know and I don't need
to know, but if folks are willing
570
00:33:30,524 --> 00:33:33,884
to share, happy to hear you out
and have a conversation about it.
571
00:33:33,954 --> 00:33:38,754
But exactly how those work because
I know many agencies are wanting to
572
00:33:38,754 --> 00:33:43,644
employ them and use them in that kind
of support sense and not the outright
573
00:33:43,644 --> 00:33:45,774
augmentation or supplementation sense.
574
00:33:46,154 --> 00:33:47,474
That we referenced earlier.
575
00:33:47,594 --> 00:33:50,864
It's just how do we skin that
onion and how do we necessarily
576
00:33:50,864 --> 00:33:52,094
get to the core of the matter?
577
00:33:52,214 --> 00:33:55,124
Yeah, I'm, I'm sure that
there's most depart.
578
00:33:55,124 --> 00:33:57,044
I shouldn't say even most
'cause I don't even know.
579
00:33:57,044 --> 00:33:59,504
I mean there's definitely just
like anything else, I mean there's
580
00:33:59,504 --> 00:34:01,274
probably just a flat out band.
581
00:34:02,804 --> 00:34:08,504
Like absolutely you are not the use jet
TBT, or you are not the use AI at all.
582
00:34:08,924 --> 00:34:13,574
Until we have a better understanding of
how all the ramifications are gonna work.
583
00:34:13,604 --> 00:34:14,054
Right.
584
00:34:14,054 --> 00:34:17,414
And it's gonna, it's gonna,
it is gonna take a little bit.
585
00:34:17,594 --> 00:34:22,694
And so, but I, I thought, man, it just,
and, and I talked to this about with
586
00:34:22,694 --> 00:34:27,684
Ollie, I just, man, I just, that's
such a when you talk about large
587
00:34:27,684 --> 00:34:33,894
language models and that aspect of it,
machine learning and that, that's a
588
00:34:33,894 --> 00:34:36,714
whole, almost a whole division within.
589
00:34:37,304 --> 00:34:39,704
Your IT department at
your police department.
590
00:34:39,974 --> 00:34:40,394
It is,
591
00:34:40,484 --> 00:34:40,874
right.
592
00:34:40,874 --> 00:34:42,944
That's a huge undertaking.
593
00:34:43,094 --> 00:34:47,714
That's not just something like, Hey,
crime analyst, solo crime analyst.
594
00:34:47,744 --> 00:34:53,894
Go, go build us a large language model on
top of all the other hats that you wear.
595
00:34:55,234 --> 00:34:59,414
I mean, that would be very, and just
to be kind of flippant, but humorous,
596
00:34:59,414 --> 00:35:03,044
it would, it would be very on-brand
for an agency to look at an analyst and
597
00:35:03,044 --> 00:35:07,454
go, Hey, you got Intel in your title,
or you got analyst in your title.
598
00:35:07,664 --> 00:35:08,564
Go do this.
599
00:35:08,654 --> 00:35:08,989
You know what I mean?
600
00:35:09,489 --> 00:35:09,549
Yeah.
601
00:35:10,189 --> 00:35:15,559
As we finish up here, and this is
obviously a big, big, big topic and it's,
602
00:35:15,559 --> 00:35:20,539
it, it is always interesting to talk
to you and get your perspective, Randy.
603
00:35:20,589 --> 00:35:22,419
, Just to finish up here.
604
00:35:22,524 --> 00:35:24,354
Before you leap, you should what?
605
00:35:24,554 --> 00:35:25,454
, Before you leap.
606
00:35:25,544 --> 00:35:29,774
You should always consider that
AI is a tool and it's not an
607
00:35:29,774 --> 00:35:34,964
outright replacement for baseline
critical thinking, trade, craft, or
608
00:35:34,964 --> 00:35:37,484
practices in our intelligence work.
609
00:35:37,574 --> 00:35:38,264
Very good.
610
00:35:38,264 --> 00:35:41,024
Randy, appreciate you as always.
611
00:35:41,354 --> 00:35:42,824
Thank you so much and you be safe.
612
00:35:43,004 --> 00:35:44,126
Hey, you too, Jason.
613
00:35:44,126 --> 00:35:46,466
Thank you for making it to
the end of another episode of
614
00:35:46,466 --> 00:35:47,846
Analyst Talk with Jason Elder.
615
00:35:47,936 --> 00:35:51,386
You can show your support by sharing
this in other episodes found on
616
00:35:51,386 --> 00:35:55,646
our website@www.leapodcasts.com.
617
00:35:55,856 --> 00:35:59,006
If you have a topic you would like
us to cover or have a suggestion
618
00:35:59,006 --> 00:36:03,716
for our next guest, please send
us an email@podcastsatgmail.com.
619
00:36:03,806 --> 00:36:05,636
Till next time, analysts, keep talking.