TRANSCRIPTIE 🔗
WEBVTT
1
00:00:04.879 --> 00:00:09.320
Welcome listeners to the Industrial Security Podcast. My name is
2
00:00:09.400 --> 00:00:13.039
Nate Nelson. I'm here with Andrew Ginter, the vice president
3
00:00:13.160 --> 00:00:17.440
of Industrial Security at Waterfall Security Solutions, who's going to
4
00:00:17.480 --> 00:00:25.519
introduce the subject and guest of our show today. Andrew,
5
00:00:25.839 --> 00:00:26.399
how's it going.
6
00:00:27.000 --> 00:00:29.839
I'm doing very well, Thank you, Nate. Our guest today
7
00:00:29.920 --> 00:00:33.159
is Marcel Rickson. He is the founder and lead instructor
8
00:00:33.200 --> 00:00:37.719
at fox Grid International, and our topic is hardware hacking.
9
00:00:37.920 --> 00:00:42.759
Picking apart the hardware, finding the vulnerabilities, you know, arguably
10
00:00:43.000 --> 00:00:45.640
essential attack knowledge. We need to understand how we're going
11
00:00:45.679 --> 00:00:48.560
to be attacked if we're going to design effective defenses.
12
00:00:48.640 --> 00:00:50.439
So that's that's the topic for today.
13
00:00:51.240 --> 00:00:55.000
Then, without further Ado, here's your conversation with Marcel.
14
00:00:57.799 --> 00:01:01.280
Hello Marcel, and welcome to the podcast. Before we get started,
15
00:01:01.280 --> 00:01:03.359
can I ask you to introduce yourself. Please tell our
16
00:01:03.359 --> 00:01:05.680
listeners a little bit about your background and about the
17
00:01:05.680 --> 00:01:07.400
good work that you're doing at Fox Grid.
18
00:01:07.959 --> 00:01:10.519
Yeah, thank you, Andrew. Hi everyone, my name is MARSA
19
00:01:10.599 --> 00:01:13.560
Riksen and if it would introduce me in one sentence,
20
00:01:13.719 --> 00:01:17.959
I am an automation engineer turned ot security nerd. To
21
00:01:18.000 --> 00:01:21.439
my background, I have a master's in automation engineering. I
22
00:01:21.480 --> 00:01:25.359
have global experience in commissioning automation systems as well as
23
00:01:25.760 --> 00:01:31.159
programming planning industrial operations. Now, during my day job, I
24
00:01:31.200 --> 00:01:35.280
am an OT and IoT security consultant and a product
25
00:01:35.359 --> 00:01:39.959
owner of our in houset remote access solution. During my nighttime,
26
00:01:40.040 --> 00:01:42.040
I am an hacker, or if you want to put
27
00:01:42.079 --> 00:01:45.920
it more formal, I am an independent OT security researcher
28
00:01:45.959 --> 00:01:50.480
that looks at what makes and breaks OT devices. Coming
29
00:01:50.519 --> 00:01:54.000
from that, I also founded Foxgrid, where I want to
30
00:01:54.079 --> 00:01:58.560
teach industrial cybersecurity and safety to newcomers.
31
00:01:59.439 --> 00:02:03.400
Thank you for that. And our topic is hardware hacking.
32
00:02:03.920 --> 00:02:05.920
Can we start with an example. You've got a couple
33
00:02:05.959 --> 00:02:07.799
of reports out, Can you pick one? Can you tell
34
00:02:07.879 --> 00:02:10.520
us about, you know, a concrete example of what what
35
00:02:10.759 --> 00:02:11.120
is that?
36
00:02:11.919 --> 00:02:15.240
Yeah, let's talk about the hardware hacking that led to
37
00:02:15.280 --> 00:02:18.280
a CVE that I found last year where I found
38
00:02:18.400 --> 00:02:22.759
hard coded root credentials hidden deep in the device's filmware memory.
39
00:02:23.520 --> 00:02:25.360
Can you go a little deeper? What was the device,
40
00:02:25.439 --> 00:02:28.879
how's it supposed to work? And you know how important
41
00:02:28.960 --> 00:02:29.680
is what you found?
42
00:02:30.439 --> 00:02:33.840
So the device is a remote access skateway that machine
43
00:02:33.879 --> 00:02:38.159
builders usually built into the electric cabinet so which connects
44
00:02:38.199 --> 00:02:41.879
the machine to the service provider. In case there's an
45
00:02:42.000 --> 00:02:47.759
unplanned interruption or any other operational bug coming up, the
46
00:02:47.800 --> 00:02:51.719
service provider can directly connect over the cloud portal to
47
00:02:51.919 --> 00:02:54.400
this edge device and start troubleshooting.
48
00:02:55.639 --> 00:02:58.439
If I made. This is something that's used in manufacturing.
49
00:02:58.439 --> 00:03:01.360
When you say the manufacturer, you mean someone who's building
50
00:03:01.400 --> 00:03:04.319
a robot, someone who's building a stamping machine, someone who's
51
00:03:04.319 --> 00:03:07.520
building I don't know, conveyor. Is that the use case here?
52
00:03:08.159 --> 00:03:12.759
This basically can be used in any operational so from
53
00:03:12.800 --> 00:03:17.439
your maybe water treatment planned to your manufacturing to your
54
00:03:17.439 --> 00:03:20.759
building automation. There are really no limits. This is really
55
00:03:20.800 --> 00:03:25.680
a network connection from the service engineer's laptop directly into
56
00:03:25.759 --> 00:03:27.919
the heart of the device or into the heart of
57
00:03:27.960 --> 00:03:28.560
the operation.
58
00:03:29.680 --> 00:03:32.639
Okay, So it's not just used for like a robot
59
00:03:32.759 --> 00:03:36.319
for a manufacturer of equipment. It might also be used
60
00:03:36.319 --> 00:03:39.759
by a service provider, by the engineer who's responsible for
61
00:03:40.759 --> 00:03:44.159
you know, occasionally coming in and servicing parts of a
62
00:03:44.240 --> 00:03:47.599
water treatment system. It's used to access systems as well
63
00:03:47.639 --> 00:03:49.479
as devices, is what I'm hearing.
64
00:03:49.840 --> 00:03:55.000
Yes, correct, So this acts as the gateway to the
65
00:03:55.039 --> 00:03:57.039
machine or operational network.
66
00:03:57.800 --> 00:04:01.199
So you've found the default credentials. Does that mean that
67
00:04:01.240 --> 00:04:04.479
any fool who wants to can connect to the cloud
68
00:04:04.520 --> 00:04:07.360
connect into this thing, or how would you use those
69
00:04:07.360 --> 00:04:09.360
default credentials.
70
00:04:09.599
Load full transcript
--> 00:04:14.120
Luckily, the attack vector is really narrow. These default credentials.
71
00:04:14.199 --> 00:04:17.720
They grant root access to the device, and you only
72
00:04:17.759 --> 00:04:21.000
can get root access when you are physically connected to
73
00:04:21.120 --> 00:04:25.519
the device. So luckily the cloud attack surface or the
74
00:04:25.560 --> 00:04:28.120
cloud is not exposed to disvulnerability.
75
00:04:31.600 --> 00:04:33.399
Andrew, I don't know if I just missed it, but
76
00:04:33.720 --> 00:04:36.879
what is the actual device that we're talking about here?
77
00:04:38.079 --> 00:04:42.000
It is an Ixon device I XO N I forget
78
00:04:42.000 --> 00:04:44.399
the exact name of it, but it's it's physically it's
79
00:04:44.439 --> 00:04:47.079
a little device about you know, six inches square in
80
00:04:47.120 --> 00:04:48.040
an inch thick.
81
00:04:49.279 --> 00:04:49.680
And.
82
00:04:51.040 --> 00:04:54.480
In my understanding, it's you know, it's a remote access device.
83
00:04:54.959 --> 00:04:59.639
You can connect into it from the cloud. Who uses this?
84
00:05:00.839 --> 00:05:03.720
The sense I have is that it's used in manufacturing.
85
00:05:03.759 --> 00:05:08.199
If you're building a laser cutter or a stamping machine,
86
00:05:08.560 --> 00:05:11.439
you might build one of these into the thing so
87
00:05:11.480 --> 00:05:13.639
that when the customer calls you up and says your
88
00:05:13.680 --> 00:05:17.360
machine isn't working, something is worn out, you can remote
89
00:05:17.399 --> 00:05:20.600
into it, do the diagnostics and say I think it's
90
00:05:20.600 --> 00:05:24.000
this part, replace this part, see if the problem is solved,
91
00:05:24.399 --> 00:05:26.839
because you know, moving parts where out friction is the
92
00:05:27.120 --> 00:05:30.279
is the enemy of moving parts. But you know, when
93
00:05:30.279 --> 00:05:36.000
I asked the gentleman Rick, he said, yeah, manufacturers of
94
00:05:36.240 --> 00:05:40.079
physical equipment use it so they can maintain the equipment
95
00:05:40.160 --> 00:05:44.279
or diagnose the equipment remotely. But it's remote access. You know,
96
00:05:44.360 --> 00:05:47.720
a service provider, an engineer who's responsible for, you know,
97
00:05:48.319 --> 00:05:51.879
keeping the automation running at a dozen small water utilities
98
00:05:51.959 --> 00:05:55.360
in the geography might well buy, you know, a half
99
00:05:55.399 --> 00:05:57.959
dozen of these and drop one of them into each
100
00:05:58.079 --> 00:06:02.079
water system to access the HMI and the automation and whatnot.
101
00:06:02.279 --> 00:06:04.560
So it's remote access. The sense I have that it's
102
00:06:04.639 --> 00:06:10.759
used frequently by manufacturers of equipment that's used in manufacturing,
103
00:06:11.000 --> 00:06:14.079
but it could be used by service providers as well.
104
00:06:14.920 --> 00:06:17.399
And I think it's the remote access thing that has
105
00:06:17.480 --> 00:06:21.759
me a little bit confused here. We're talking about hard
106
00:06:21.800 --> 00:06:26.199
coded credentials as vulnerability, something I'm rather used to in
107
00:06:26.319 --> 00:06:30.480
the IT space, right, Like a public repository or a
108
00:06:30.560 --> 00:06:37.040
server that's been incorrectly configured will leak credentials to the
109
00:06:37.040 --> 00:06:39.680
web that then actors could use to get in and
110
00:06:39.720 --> 00:06:44.360
we're talking about a remote access device, and yet I
111
00:06:44.360 --> 00:06:47.920
think he mentioned there that you can only actually exploit
112
00:06:48.000 --> 00:06:53.079
this vulnerability if you have local physical access to the machine.
113
00:06:53.399 --> 00:06:56.240
So can you help me explain that gap.
114
00:06:57.240 --> 00:06:59.079
We go into this in sort of more detail later
115
00:06:59.120 --> 00:07:01.600
in the interview, but let me let me let people
116
00:07:01.600 --> 00:07:05.560
know kind of what's happening. This is you know, there
117
00:07:05.560 --> 00:07:10.319
are basically two user interfaces to the device. One is
118
00:07:10.720 --> 00:07:13.680
the remote access user interface with users configured and blah
119
00:07:13.720 --> 00:07:19.160
blah blah. That's not where the vulnerability is. The other
120
00:07:19.279 --> 00:07:22.319
interface is if you touch the device and you connect
121
00:07:22.319 --> 00:07:24.040
to it. I don't know, I was a little weak
122
00:07:24.079 --> 00:07:26.920
on the details. If you connect through the USB port
123
00:07:27.040 --> 00:07:30.839
or if you connect you know pins, you know, you're
124
00:07:31.040 --> 00:07:34.279
too electrically to pins sitting there on the on the
125
00:07:34.439 --> 00:07:36.480
on the circuit board. If you open the device up,
126
00:07:37.360 --> 00:07:41.319
you can get access to the operating system of the device.
127
00:07:42.079 --> 00:07:45.240
And it's the operating system credentials that were lead so
128
00:07:45.279 --> 00:07:48.000
in order to use those credentials, they don't work on
129
00:07:48.079 --> 00:07:53.680
the remote user interface. They work locally when you're when
130
00:07:53.680 --> 00:07:56.319
you're able to physically touch the device and plug stuff
131
00:07:56.360 --> 00:08:00.399
into it. The CVE was twenty twenty four, dash five seven,
132
00:08:00.600 --> 00:08:04.079
seven nine zero. It was given a five or a
133
00:08:04.120 --> 00:08:06.240
five point nine or something like this, not a ten.
134
00:08:06.360 --> 00:08:09.800
This is not a remote code execution vulnerability. You can't
135
00:08:09.800 --> 00:08:11.759
do this remotely. You have to be local. It's a
136
00:08:11.839 --> 00:08:15.199
local escalation of privilege vulnerability.
137
00:08:15.680 --> 00:08:17.879
That explanation makes a lot of sense to me, But
138
00:08:18.079 --> 00:08:20.959
why is it that, Like, how can you even leak
139
00:08:21.040 -->
00:08:25.560
credentials to somebody who's physically using a computer? Right, Like,
140
00:08:25.800 --> 00:08:28.439
any credentials on my computer that get leaked to me
141
00:08:28.839 --> 00:08:31.639
doesn't matter because I'm the user. So what I suppose
142
00:08:31.680 --> 00:08:36.159
I'm asking attack scenarios are we worried about with this vulnerability.
143
00:08:37.120 --> 00:08:39.279
Well, actually we didn't go into that, but you know,
144
00:08:39.360 --> 00:08:43.679
as far as I know, the scenario is that you're
145
00:08:43.679 --> 00:08:47.399
there locally touching the device. Now normally, you know, you
146
00:08:47.440 --> 00:08:50.440
look at the device, it's got network ports, it's got
147
00:08:50.879 --> 00:08:53.679
you know, one of the ports is connected out to
148
00:08:54.120 --> 00:08:57.320
the world. You come in remotely, you know it does
149
00:08:57.320 --> 00:09:03.080
its thing. There is no other supported user interface. But
150
00:09:03.559 --> 00:09:05.360
if you touch the device, you can get in there.
151
00:09:05.399 --> 00:09:08.240
You can tamper with the firmwaire, you can tamper with stuff.
152
00:09:08.279 --> 00:09:13.320
You can you know, you could presumably create credentials that
153
00:09:13.360 --> 00:09:15.559
you could use remotely. You'd need a little bit of
154
00:09:15.600 --> 00:09:18.480
skill to do that, but you know you could break
155
00:09:18.519 --> 00:09:22.159
the device, so it's again if you're standing there with
156
00:09:22.200 --> 00:09:24.519
a hammer, you could also break the device. This is
157
00:09:24.559 --> 00:09:27.519
why it was given a lower priority. Yes, technically it's
158
00:09:27.519 --> 00:09:34.240
a vulnerability, it's not a really alarming one. What's interesting
159
00:09:34.399 --> 00:09:37.480
is how did you find it? Because the way that
160
00:09:37.559 --> 00:09:40.480
he found it, the technique is what he teaches at
161
00:09:40.519 --> 00:09:44.519
Foxgrid you can also use to find more interesting stuff.
162
00:09:47.120 --> 00:09:48.679
What I wanted to ask you about is, you know,
163
00:09:48.720 --> 00:09:54.960
we've never had someone on the show who picks things
164
00:09:54.960 --> 00:09:57.960
apart like this, or maybe we did once about three
165
00:09:58.000 --> 00:10:00.759
years ago, but it's been a long time, you know.
166
00:10:00.919 --> 00:10:05.120
Can you talk about the process? How did you find this?
167
00:10:05.320 --> 00:10:08.879
How does one pick these things apart? What is what
168
00:10:08.919 --> 00:10:09.559
does that mean?
169
00:10:10.399 --> 00:10:13.000
If I would just describe it to you maybe in
170
00:10:13.000 --> 00:10:16.240
a pub or over our coffee. This really is like
171
00:10:16.320 --> 00:10:19.720
a hardware and digital scavenger hunt because you have to
172
00:10:19.759 --> 00:10:22.360
look at so many things. You also go down a
173
00:10:22.399 --> 00:10:24.960
rabbit hole, see it's the wrong way, turn around, and
174
00:10:25.000 --> 00:10:29.399
then keep digging. And to find this I just needed
175
00:10:29.879 --> 00:10:37.039
tools for about thirty euros, so a multimeter, screwdriver, prying tools,
176
00:10:37.679 --> 00:10:41.320
a U s BI logic analyzer, and a usbut interface
177
00:10:42.039 --> 00:10:44.480
was all I needed to find this ulnerability.
178
00:10:45.480 --> 00:10:47.600
Can you give me a little more detail? Those are
179
00:10:47.639 --> 00:10:51.200
the parts. I've never used a logic analyzer. I mean,
180
00:10:51.480 --> 00:10:53.240
how technical is that? Do I need to be an
181
00:10:53.279 --> 00:10:56.080
engineer to use a logic analyzer? How did? How did
182
00:10:56.120 --> 00:10:58.840
you go about this? What you know? Can you tell
183
00:10:58.919 --> 00:11:01.720
us a story? You know, what did you start with?
184
00:11:01.840 --> 00:11:04.279
What do you do next? What does that mean? And
185
00:11:04.720 --> 00:11:08.159
you know a blind alley went down? How did it work?
186
00:11:08.960 --> 00:11:11.240
Yeah? I can walk you through the all the six
187
00:11:11.360 --> 00:11:15.399
or seven steps that led to root access, from opening
188
00:11:15.480 --> 00:11:19.039
up to the device until I was greeted with the
189
00:11:19.120 --> 00:11:26.600
root banner. Okay, And before anyone gets started with hardware
190
00:11:26.639 --> 00:11:30.440
hacking and picking a dior, before anyone gets started with
191
00:11:30.600 --> 00:11:35.159
taking a device apart, here are four electrical safety rules
192
00:11:35.200 --> 00:11:38.759
that you should follow because your own life is more
193
00:11:38.799 --> 00:11:45.320
important than your curiosity. And never ever open wall plucked
194
00:11:45.399 --> 00:11:49.440
devices because if they're directly plucked into the socket, this
195
00:11:49.639 --> 00:11:55.120
means that hazardous voltage is inside the PCB, inside the device,
196
00:11:55.200 --> 00:11:58.480
and there's the risk that you can touch a live
197
00:11:58.559 --> 00:12:04.759
wire and it's urisk and electric shock. Therefore, use devices
198
00:12:04.799 --> 00:12:08.200
that have external power adapters only so that the voltage
199
00:12:08.240 --> 00:12:13.200
conversion happens outside the area where you're working with. Also
200
00:12:13.279 --> 00:12:18.720
avoid mixing power sources. This is important for example if
201
00:12:18.759 --> 00:12:24.360
you really go into firmware extraction. And of course prevent
202
00:12:24.440 --> 00:12:27.960
in short circuits because this will fry your PCB and
203
00:12:28.000 --> 00:12:31.639
then you have a very expensive brick. But if you
204
00:12:31.639 --> 00:12:34.639
stick to these rules, you can open up the device
205
00:12:34.759 --> 00:12:39.200
and first start with a hardware reconnaissance. We'll just take
206
00:12:39.240 --> 00:12:43.960
a look on what chips are on there. And many
207
00:12:44.000 --> 00:12:47.720
industrial embedded devices
they'll run on a so called system
208
00:12:47.840 --> 00:12:51.000
on chip and they are somewhere close to the system
209
00:12:51.039 --> 00:12:56.440
on chip the flash memory filmware chip. So this is
210
00:12:56.480 --> 00:13:00.519
basically where the brains and oil information are still and
211
00:13:00.799 --> 00:13:03.759
once it is and once the system is powered on,
212
00:13:04.399 --> 00:13:08.279
the system on chip pulls the filmware information from the
213
00:13:08.320 --> 00:13:13.480
firmware memory chip. If you identify these, then you take
214
00:13:13.519 --> 00:13:17.120
a look around the board. Are there any debug interfaces
215
00:13:17.320 --> 00:13:20.799
and and on this device I found the so called
216
00:13:21.159 --> 00:13:27.840
Ewert debugging interface. So with these we can move to
217
00:13:27.879 --> 00:13:31.960
the next steps. And first we do some electrical measurements
218
00:13:32.320 --> 00:13:35.879
just to prevent that our us be US BWARED and
219
00:13:36.039 --> 00:13:40.519
USB Signal analyzers get fried because they are very sensitive
220
00:13:40.559 --> 00:13:46.039
to voltage. And first things first, we first confirm the
221
00:13:46.360 --> 00:13:51.440
common electrical ground on the debug interface we identified. Once
222
00:13:51.440 --> 00:13:54.600
we identify the common ground, we know where we well
223
00:13:54.600 --> 00:13:59.960
connect the ground wire of our USB logic analyzer. Then
224
00:14:00.120 --> 00:14:04.840
a viewered interface usually has two more pins RX and TX,
225
00:14:05.039 --> 00:14:10.720
which stands for receive and transmit. And then we turn
226
00:14:10.799 --> 00:14:14.960
on the power and then measure these pins against the
227
00:14:15.279 --> 00:14:20.480
electrical common ground and in most cases we will find
228
00:14:20.519 --> 00:14:25.480
a voltage range between three walls and five walls, which
229
00:14:25.519 --> 00:14:29.360
means the devices are communicating on the so called transistor
230
00:14:29.399 --> 00:14:34.440
transistor logic level. When this is identified, we can move
231
00:14:34.519 --> 00:14:38.960
on with the logic analyzer, power off the device, connect
232
00:14:38.960 --> 00:14:42.320
the logic analyzer. We connect the logic analyzer's ground to
233
00:14:42.399 --> 00:14:46.679
the boards ground, and then the RX and TX WIA.
234
00:14:47.120 --> 00:14:52.480
Although the RX and TX pin are already labeled and
235
00:14:52.519 --> 00:14:56.120
we only could connect to the TX, it's always good
236
00:14:56.120 --> 00:14:59.639
to connect to all pins that we have a full
237
00:14:59.679 --> 00:15:03.000
picture of what's going on. Because Andrew at this board
238
00:15:03.120 --> 00:15:08.039
is what's this what's easy mode? The pins are already labeled,
239
00:15:08.080 --> 00:15:11.000
but that's not always the case sometimes you just have
240
00:15:11.120 --> 00:15:14.679
a three four five pin sticking out and you don't
241
00:15:14.720
--> 00:15:18.000
know what I mean. Then you also do the same procedure.
242
00:15:18.080 --> 00:15:21.600
You measure for the electric common ground and then start
243
00:15:21.639 --> 00:15:25.039
measuring the voyage levels, and this gives you an idea
244
00:15:25.840 --> 00:15:28.879
if you can find logical signals going on.
245
00:15:30.000 --> 00:15:32.279
So thanks for that. I mean, that's you know, giving
246
00:15:32.399 --> 00:15:35.600
giving us some insight into the mechanics of dealing with
247
00:15:35.600 --> 00:15:38.759
the device. I mean, I'm I'm a software guy. I
248
00:15:38.840 --> 00:15:42.120
never have to worry about electrocuting myself if if I
249
00:15:42.200 --> 00:15:47.639
bring up a compiler on my laptop. But let me
250
00:15:47.679 --> 00:15:51.320
ask you know, you've talked about two sort of devices
251
00:15:51.360 --> 00:15:55.679
here that seem to ring a bell with me. There's
252
00:15:55.759 --> 00:16:01.600
the the you aret, which a quick question is that
253
00:16:01.679 --> 00:16:04.799
you are USB or is it RS two thirty two.
254
00:16:05.960 --> 00:16:11.600
This is an RS arras thirty two connection to USB.
255
00:16:11.759 --> 00:16:17.120
So this basically converts the signal, converts the logic serial
256
00:16:17.240 --> 00:16:21.399
signals to USB so that your machine, your computer can
257
00:16:21.519 --> 00:16:22.039
work with that.
258
00:16:25.120 --> 00:16:28.120
Nate real quick here, I had a very short sort
259
00:16:28.120 --> 00:16:32.519
of interaction with with Rick there asking about the difference
260
00:16:32.559 --> 00:16:34.960
between USB and RS two thirty two. For anyone who
261
00:16:35.039 --> 00:16:38.320
didn't quite track that RS two thirty two is a
262
00:16:38.559 --> 00:16:42.080
very old hardware signaling protocol. I mean I remember using
263
00:16:42.200 --> 00:16:45.159
RS two thirty two back in the day to connect
264
00:16:45.200 --> 00:16:47.120
you know, this was thirty years ago to connect to
265
00:16:47.320 --> 00:16:52.480
three hundred bits per second modems. Okay, ancient ancient technology.
266
00:16:52.679 --> 00:16:55.480
Why would there be such an ancient interface on this
267
00:16:55.600 --> 00:16:58.360
modern device, is roughly what I asked him, and he said,
268
00:16:58.399 --> 00:17:01.600
there isn't. What there is is is a USB port.
269
00:17:01.840 --> 00:17:03.919
It turns out that what he connected to, the t
270
00:17:04.240 --> 00:17:08.839
X and RX he connected to were signaling USB. And
271
00:17:09.160 --> 00:17:12.119
when he looked at the signals, he discovered that the
272
00:17:12.200 --> 00:17:16.759
messages coming across the USB were r S two thirty
273
00:17:16.759 --> 00:17:19.880
two over USB. So he looked around. He said, well,
274
00:17:20.000 --> 00:17:24.160
you know, I have a dongle that can that you know,
275
00:17:24.200 --> 00:17:26.480
can take USB and gives me RS two thirty two,
276
00:17:26.519 --> 00:17:29.079
and he connected to it and there he can see
277
00:17:29.119 --> 00:17:33.079
the messages coming across. So that's that's what's going on
278
00:17:33.160 --> 00:17:35.920
there is it's a US thirty two thirty it's a
279
00:17:36.039 --> 00:17:41.680
USB connector on the device, but the signaling is r
280
00:17:41.759 --> 00:17:47.359
S two thirty two over USB. The other one that
281
00:17:47.519 --> 00:17:50.960
that struck me, and again I'm a software guy. You
282
00:17:51.079 --> 00:17:55.119
mentioned the flash chip. You know. To me, if I
283
00:17:55.200 --> 00:17:58.359
get what's on the flash, I can start looking at instructions,
284
00:17:58.400 --> 00:18:02.519
I can start running my business. Embler, Is it possible
285
00:18:02.640 --> 00:18:05.240
to to sort of go under the nose of the
286
00:18:05.240 --> 00:18:08.480
device and just read the flash chip? Or do you
287
00:18:08.519 --> 00:18:09.759
have to go through the front door? Do you have
288
00:18:09.799 --> 00:18:11.839
to go through the CPU in order to get access
289
00:18:11.839 --> 00:18:13.920
to the flash? No, you don't.
290
00:18:13.960 --> 00:18:17.240
You also can basically perform a chip off of the
291
00:18:17.279 --> 00:18:21.119
flash chip and then read out the contents with a programmer.
292
00:18:22.119 --> 00:18:25.960
This is also possible, but at the time of my research,
293
00:18:26.039 --> 00:18:28.960
I didn't have such equipment here, so I went through
294
00:18:29.000 --> 00:18:29.640
the front door.
295
00:18:30.640 --> 00:18:33.319
Okay, that's fair, So please carry on. You're you're talking
296
00:18:33.319 --> 00:18:36.279
about the u R. I interrupted you. You know, finish
297
00:18:36.440 --> 00:18:38.000
finish the story here. How how are we?
298
00:18:38.160 --> 00:18:38.279
Uh?
299
00:18:38.960 --> 00:18:39.759
How did you get in?
300
00:18:40.440 --> 00:18:44.319
Okay? The logic analyser revealed that indeed, logical data is
301
00:18:44.519 --> 00:18:48.359
transmitted over the t xpin. So this means we can
302
00:18:48.559 --> 00:18:53.279
connect our us BURET interface and open a serial console
303
00:18:53.400 --> 00:18:54.720
to that.
304
00:18:54.720 --> 00:18:57.160
That's fair. I didn't realize it USB will I mean,
305
00:18:57.200 --> 00:18:59.759
I knew USB was cereal, that's what it means, universal
306
00:19:00.119 --> 00:19:05.400
real bus, but I guess I never put two and
307
00:19:05.440 --> 00:19:08.440
two together that you could just, you know, I don't know,
308
00:19:08.480 --> 00:19:10.920
connect an RS two thirty two to it.
309
00:19:12.000 --> 00:19:15.319
Well, you always need this interface device that you pluck
310
00:19:15.440 --> 00:19:18.759
between that you plug into your USB socket and then
311
00:19:18.920 --> 00:19:21.400
on the other end of the device you can connect
312
00:19:21.440 --> 00:19:25.559
it to the target device. So once the usbu WED
313
00:19:25.599 --> 00:19:29.680
interface is connected, and I started the
terminal on and
314
00:19:29.720 --> 00:19:33.119
I started a serial console on my Linux machine. Then
315
00:19:33.160 --> 00:19:36.960
I powered up the device again. I could see the
316
00:19:37.119 --> 00:19:41.079
bootlock flashing in front of the screen. You know, it
317
00:19:41.160 --> 00:19:45.160
was a basic Linux lute bop. It was a very
318
00:19:45.759 --> 00:19:48.799
It was a basic Linux bootlock, and at the very
319
00:19:48.920 --> 00:19:51.640
end there was a lock in prompt to lock into
320
00:19:51.680 --> 00:19:55.319
this device. This is where it got interesting. And here
321
00:19:55.400 --> 00:19:59.119
my curiosity was really on fire because I really wanted
322
00:19:59.119 --> 00:20:01.559
to get into this d and I started to look
323
00:20:01.599 --> 00:20:06.000
at the bootlock itself. First. Here I learned that the
324
00:20:06.039 --> 00:20:11.200
filmware memory is partitioned into several partitions. And if you
325
00:20:11.279 --> 00:20:16.200
look at the common IoT hardware hacker courses, then they
326
00:20:16.240 --> 00:20:18.839
always tell you to go for the root FS file
327
00:20:18.920 --> 00:20:22.839
system because that's where all the binaries are stored off
328
00:20:22.839 --> 00:20:26.400
this Linux device. But there was another petition that was
329
00:20:26.559 --> 00:20:29.920
interesting for me. This was the so called factory partition.
330
00:20:31.119 --> 00:20:34.119
Scrawling further up in the bootlock, there was also a
331
00:20:34.160 --> 00:20:37.799
brief prompt to pass a There was also a brief
332
00:20:37.839 --> 00:20:42.160
prompt to press space bar to enter the bootloader, but
333
00:20:42.319 --> 00:20:45.000
andrew the timing for this was so narrow that it
334
00:20:45.079 --> 00:20:48.200
was almost impossible to hit the timing right to enter
335
00:20:48.240 --> 00:20:52.039
the bootloader. You know, you can imagine I was jamming
336
00:20:52.119 --> 00:20:55.160
the I was hammering the space bar like a lunatic.
337
00:20:56.079 --> 00:20:58.960
And then maybe at the fourth or fifth time, I
338
00:20:59.000 --> 00:21:01.720
succeeded to get the timing right, and then I was
339
00:21:02.000 --> 00:21:06.200
presented with the next option to choose the operation. And
340
00:21:06.279 --> 00:21:10.440
here a very interesting option was presented to me. By
341
00:21:10.440 --> 00:21:13.039
pressing the number four, I would be able to enter
342
00:21:13.759 --> 00:21:17.759
the boot command line interface. And this was something what
343
00:21:17.799 --> 00:21:20.519
I was interested in and wanted to go. But with
344
00:21:20.880 --> 00:21:26.960
this narrow timing, I turned to chat GPT, asking it
345
00:21:27.000 --> 00:21:29.400
is this way. Is there a way that I can
346
00:21:29.480 --> 00:21:33.440
automate the keypresses and can send a space bar press
347
00:21:33.480 --> 00:21:37.519
and a number four press at rapid speed? The AI
348
00:21:38.240 --> 00:21:43.720
gave me a five line
shell script code which uses
349
00:21:44.359 --> 00:21:47.759
an on board tool of Kelly Linux to send space
350
00:21:47.839 --> 00:21:51.759
bar and number fours, and this immediately lended me into
351
00:21:51.799 --> 00:21:57.000
the bootshell. And the bootshell of this device is based
352
00:21:57.039 --> 00:22:01.000
on the U boot bootloader, and all the hardware hackers
353
00:22:01.000 --> 00:22:03.480
out there that are familiar with you boot would immediately
354
00:22:03.519 --> 00:22:05.960
see that this is already a very stripped down and
355
00:22:06.599 --> 00:22:12.200
secured restricted version of you Boot. There was almost no
356
00:22:12.359 --> 00:22:16.519
way of manipulating the device, but they're left in the
357
00:22:16.559 --> 00:22:20.480
so called SPI command which enabled me to read the
358
00:22:20.519 --> 00:22:24.279
content of the factory partition. So that's what I did.
359
00:22:24.279 --> 00:22:28.759
I issued the command to read the factory petition and
360
00:22:28.799 --> 00:22:32.440
then and then it printed out the content of the
361
00:22:32.480 --> 00:22:37.599
factory petition in the hexadecimal format. And here's something really
362
00:22:37.640 --> 00:22:41.319
strange occurred to me that the data was not always
363
00:22:41.400 --> 00:22:47.279
represented in two hexadecimal digits. You know that, You know
364
00:22:47.359 --> 00:22:51.720
hexadecimal data always needs to have two digits. If not,
365
00:22:51.920 --> 00:22:57.400
the data gets misaligned and then gets corrupted. So the
366
00:22:57.440 --> 00:23:00.319
problem I was facing here is that some digits where
367
00:23:00.559 --> 00:23:05.119
or some data was represented with single digits missing the
368
00:23:05.160 --> 00:23:10.880
second digits, so the data was not usable for me. Okay,
369
00:23:10.960 --> 00:23:13.880
Then I used another script to align the data and
370
00:23:13.920 --> 00:23:19.720
then convert the text has the text hexidescimal data back
371
00:23:19.759 --> 00:23:23.599
into binary hexadescimal data, and then I was able to
372
00:23:23.839 --> 00:23:28.039
view the binary data and the ask the interpretation of that.
373
00:23:28.920 --> 00:23:34.400
And here's something really interesting stood out. There were basically
374
00:23:34.680 --> 00:23:40.640
three strings of data that at first made not really
375
00:23:40.720 --> 00:23:45.200
sense to me, but somehow felt familiar. And suddenly I
376
00:23:45.240 --> 00:23:49.240
realized this is the information which is also printed on
377
00:23:49.279 --> 00:23:53.359
the device's label on the site. I could see that
378
00:23:53.400 --> 00:23:58.640
in this factory partition of this device, the version number,
379
00:23:58.920 --> 00:24:03.319
the serial number, the device version, and the lock in
380
00:24:03.400 --> 00:24:07.359
password for the web management surface for the web management
381
00:24:07.440 --> 00:24:13.279
interface was stored. But there was another string that also
382
00:24:13.359 --> 00:24:17.759
kept me guessing and puzzling for quite a while. But
383
00:24:18.079 --> 00:24:22.640
this unknown string had the same characteristics as the weblock
384
00:24:22.680 --> 00:24:26.839
in password. It had ten characters, capital and lower case
385
00:24:26.920 --> 00:24:30.119
letters and numbers, and I tell you this had to
386
00:24:30.160 --> 00:24:35.400
be another password. So I restarted the device again and
387
00:24:35.480 --> 00:24:38.079
at the very end of the boot process, I was
388
00:24:38.160 --> 00:24:41.359
prompted for the lock in once more. I entered the
389
00:24:41.480 --> 00:24:46.000
user name route and entered this data I found inside
390
00:24:46.039 --> 00:24:49.359
the memory, and this gave me root access to the device.
391
00:24:52.680 --> 00:24:55.400
Andrew, it's not that anything that Marcell said at any
392
00:24:55.400 --> 00:24:59.880
point there wasn't clear. But we've now gone a while
393
00:25:00.279 --> 00:25:03.759
and he's expressed a lot of technical steps. Can you
394
00:25:03.880 --> 00:25:08.200
just give me the big picture summary what we're talking
395
00:25:08.240 --> 00:25:10.400
about here, what he achieved and why it's important.
396
00:25:10.920 --> 00:25:15.920
Absolutely, he real quick managed to connect to the boot
397
00:25:16.079 --> 00:25:22.680
shell with the space for you know, script constantly blasting,
398
00:25:22.960 --> 00:25:24.960
and he got in and discovered there was almost nothing
399
00:25:24.960 --> 00:25:27.440
he could do there, but he could look at this
400
00:25:27.559 --> 00:25:31.000
one tiny partition and he managed to you know, get
401
00:25:31.079 --> 00:25:34.240
the data, decode the data, and he looked at he said,
402
00:25:34.240 --> 00:25:37.000
you know, this looks like a serial number. It looks
403
00:25:37.319 --> 00:25:40.480
it looks like a password. And so he said, well,
404
00:25:40.680 --> 00:25:43.160
let's try it, and he reboots the device again. He
405
00:25:43.160 --> 00:25:44.960
doesn't do the space for this time. He lets it
406
00:25:45.000 --> 00:25:47.400
completely boot and it comes up and says, okay, I'm
407
00:25:47.400 --> 00:25:49.720
ready log in. Do you want to log in? And
408
00:25:49.720 --> 00:25:54.160
he said, yeah, let's log in as root and it says, well,
409
00:25:54.160 --> 00:25:56.240
what's the root password? And he says, well, here's the
410
00:25:56.279 --> 00:25:58.400
string that I saw in the partition. He enters it
411
00:25:58.440 --> 00:26:03.680
and he's in and now he's in route. It's not
412
00:26:04.279 --> 00:26:07.519
like you looked at the file system and said, oh,
413
00:26:07.599 --> 00:26:11.519
here's files. You know, look, there's a file with the
414
00:26:11.599 --> 00:26:15.240
name password. It wasn't nearly that obvious.
415
00:26:15.720 --> 00:26:18.799
Now it was very well hidden, but I think also
416
00:26:18.880 --> 00:26:23.640
on purpose, because
there's nothing written in this memory area
417
00:26:24.200 --> 00:26:27.519
before or after this partition. You really just find the
418
00:26:28.279 --> 00:26:31.759
version number, the serial number, the web management password, and
419
00:26:32.000 --> 00:26:36.039
well the root password. So somewhere in production when the
420
00:26:36.119 --> 00:26:41.000
device gets so to speak, gets the breath of life
421
00:26:41.000 --> 00:26:45.599
and the data for the label, at this moment, the
422
00:26:45.720 --> 00:26:48.640
data must be flashed into this filmweb memory chip.
423
00:26:49.880 --> 00:26:52.519
Okay, so this is a very small partition. Then we're
424
00:26:52.559 --> 00:26:57.000
not talking, you know, tons of megabytes, We're talking tons
425
00:26:57.039 --> 00:26:57.720
of kilobytes.
426
00:26:58.480 --> 00:26:59.960
Yeah, it was very small indeed.
427
00:27:00.680 --> 00:27:06.720
Okay, Okay, cool. So you found the vulnerability and then
428
00:27:06.920 --> 00:27:10.920
I assume you know there's something called a responsible disclosure process.
429
00:27:10.960 --> 00:27:15.400
I assume you contacted the vendor, you contacted the government.
430
00:27:15.640 --> 00:27:17.519
What was the next step there?
431
00:27:18.200 --> 00:27:21.960
So the next step was to contact the security contact
432
00:27:22.359 --> 00:27:25.480
of this company, and luckily I was already in contact
433
00:27:25.839 --> 00:27:29.680
with him on LinkedIn. So on a Sunday morning, I
434
00:27:29.680 --> 00:27:32.880
sent him a screenshot of hey, mister x y Z,
435
00:27:33.079 --> 00:27:38.119
I got root access to your ot gateway and within
436
00:27:38.160 --> 00:27:41.400
two hours he replied and said, okay, this is very concerning.
437
00:27:41.880 --> 00:27:45.119
Please send your findings and everything you have to our
438
00:27:45.240 --> 00:27:48.799
security email address, and we will look into this first
439
00:27:48.799 --> 00:27:52.279
thing Monday morning. Then I wrote a quick report attached
440
00:27:52.319 --> 00:27:56.200
the screenshots and the proof of concept video, and around
441
00:27:56.559 --> 00:28:02.400
Monday lunchtime they replied said, yes, this root password is
442
00:28:02.880 --> 00:28:08.480
uniquely generated per device and inserted here during production. But
443
00:28:08.680 --> 00:28:13.599
since everything is uniquely they kind of hinted at that
444
00:28:13.640 --> 00:28:18.240
they're accepting the risks so that the probability of this
445
00:28:18.400 --> 00:28:23.680
being exploited is rather low. They also said, if machine builders,
446
00:28:23.920 --> 00:28:29.519
integrateors operators stick to their security requirements, they do not
447
00:28:29.680 --> 00:28:32.440
see really a risk of this being exploited.
448
00:28:33.759 --> 00:28:38.400
Okay, so the vendor said, you know it's a low
449
00:28:38.440 --> 00:28:43.119
priority because you know people are expected at physical security.
450
00:28:43.160 -->
00:28:45.240
No full off the street can come in, take one
451
00:28:45.240 --> 00:28:48.480
of these devices, walk away with it, pick it apart,
452
00:28:48.680 --> 00:28:52.440
bring it back. That's that's not a realistic threat. Do
453
00:28:52.519 --> 00:28:53.200
you agree with that?
454
00:28:54.039 --> 00:28:57.079
Yes, totally. I agree with that from on my experience
455
00:28:57.160 --> 00:29:00.000
on the shop floor and in the field. You cannot
456
00:29:00.279 --> 00:29:03.640
just walk up to an electric cabinet, take out their device,
457
00:29:03.799 --> 00:29:07.480
screw it open, extract the root credentials and then put
458
00:29:07.519 --> 00:29:10.039
it back in with the back door you implanted right,
459
00:29:10.440 --> 00:29:12.640
This would hopefully catch some attention.
460
00:29:14.039 --> 00:29:16.799
Okay, and you know, can you can you finish the
461
00:29:16.799 --> 00:29:18.799
the the thought? I mean, you wound up with a
462
00:29:18.839 --> 00:29:23.000
CVE for this, you know you've interacted with a vendor,
463
00:29:23.160 --> 00:29:25.039
then then what how do you how do you finish
464
00:29:25.079 --> 00:29:25.640
the process.
465
00:29:26.960 --> 00:29:31.680
Then I contacted MITRO to file a CVE, also reported
466
00:29:32.519 --> 00:29:35.519
the things I found and the implications for this, and
467
00:29:36.720 --> 00:29:39.200
after two months the CVE was.
468
00:29:39.200 --> 00:29:42.720
Assigned and at that point you're able to disclose publicly.
469
00:29:42.759 --> 00:29:43.240
Is that right?
470
00:29:43.920 --> 00:29:47.880
Yes, all that being said, there is a tiny, tiny
471
00:29:47.960 --> 00:29:52.680
risk that you may receive the backdoor device. But then
472
00:29:52.720 --> 00:29:56.319
someone really must be targeting your operations. They need to
473
00:29:56.359 --> 00:29:59.960
know that you're operating such device, and if you expe
474
00:30:00.039 --> 00:30:04.000
acting a new shipment, they could intercept the shipment, open
475
00:30:04.079 --> 00:30:08.160
up the device, extract the root credentials, implant a backdoor,
476
00:30:08.440 --> 00:30:12.039
pack it back up, and ship it forward to your operations.
477
00:30:12.680 --> 00:30:16.759
So for that, if you are operating something critical, or
478
00:30:16.799 --> 00:30:21.119
if you're operating or if you're having critical infrastructure and operations,
479
00:30:21.640 --> 00:30:26.640
you should definitely opt for temper detection and protection. You know,
480
00:30:26.720 --> 00:30:31.799
some devices they have these little sticker on their warranty
481
00:30:31.920 --> 00:30:33.880
void if removed.
482
00:30:34.799 --> 00:30:37.160
So fascinating stuff, at least at least to me. I've
483
00:30:37.160 --> 00:30:39.960
always wondered, you know, how some of this hardware hacking
484
00:30:40.039 --> 00:30:43.880
was done. But you know, as far as I know,
485
00:30:44.359 --> 00:30:47.680
you don't get paid to do the hardware hacking, unless
486
00:30:47.720 --> 00:30:50.160
I don't know there's a bounty or something. You know,
487
00:30:50.200 --> 00:30:53.440
how does this relate to making a living for you?
488
00:30:54.039 --> 00:30:56.359
This is not my day job. And I also don't
489
00:30:56.359 --> 00:30:59.599
get paid to find these fulndabilities. Let's just say this
490
00:30:59.640 --> 00:31:04.559
is a very expensive hobby. I've been in the I've
491
00:31:04.599 --> 00:31:07.599
been in the domain of automation systems for half of
492
00:31:07.640 --> 00:31:11.599
my life, and after my work, I'm still interested, especially
493
00:31:11.599 --> 00:31:15.839
in what makes and breaks these devices. And that's also
494
00:31:16.480 --> 00:31:20.119
how my trainings got born. I took all the experience
495
00:31:20.240 --> 00:31:24.200
I made from while breaking these devices and turned them
496
00:31:24.200 --> 00:31:24.880
into training.
497
00:31:26.039 --> 00:31:28.200
Okay, and you know this is what you do at
498
00:31:28.200 --> 00:31:29.960
Fox Squid. Can you go a little deeper? I mean,
499
00:31:31.599 --> 00:31:34.400
if I sign up for one of these courses, what
500
00:31:34.440 --> 00:31:35.480
are you going to lead me through?
501
00:31:36.480 --> 00:31:39.400
If we stay with hardware hacking, you could sign up
502
00:31:39.440 --> 00:31:43.640
to Industrial Embedded Systems hardware penetration testing, where you will
503
00:31:43.680 --> 00:31:47.880
also go through these six or seven steps from investigating
504
00:31:47.920 --> 00:31:52.839
the PCB to hopefully getting route access. But this course
505
00:31:52.880 --> 00:31:55.759
has a unique approach because if you look at the
506
00:31:55.839 --> 00:32:00.680
IoT hardware hacking courses, you usually hack some key camera
507
00:32:00.960 --> 00:32:05.000
or a home router, but it's almost impossible to hack
508
00:32:05.039 --> 00:32:08.640
an industrial device because there is an entry barrier problem.
509
00:32:09.240 --> 00:32:12.640
First of all, this hardware is really expensive. You usually
510
00:32:12.759 --> 00:32:16.119
pay five hundred dollars or more, and it's risky because
511
00:32:16.119 --> 00:32:19.680
you can brick it and then you wasted five hundred dollars.
512
00:32:21.000 --> 00:32:24.559
To get around this, I built a custom firmware for
513
00:32:24.680 --> 00:32:28.400
a cheap ESP eighty two sixty six micro controller that
514
00:32:28.640 --> 00:32:32.960
mimics the behavior of an industrial device and introduces the
515
00:32:33.000 --> 00:32:36.319
student to the same challenges I faced.
516
00:32:36.519 --> 00:32:39.519
Okay, so that's the hardware hacking. Have you got other courses?
517
00:32:39.880 --> 00:32:44.440
Yes? I have my flagship course, Practical Offensive Industrial Security Essentials,
518
00:32:45.240 --> 00:32:49.960
which gives students from diverse backgrounds, whether they're automation engineers,
519
00:32:50.160 --> 00:32:57.039
IT professionals
or total newcomers, a holistic introduction to industrial cybersecurity.
520
00:32:57.720 --> 00:33:00.680
Of course, there are assrom gaps that needs to be filled,
521
00:33:00.720 --> 00:33:06.599
but anyone with anyone with enough curiosity will get through
522
00:33:06.640 --> 00:33:10.359
this course or will have success with the course and
523
00:33:10.960 --> 00:33:14.480
then get a holistic understanding of industrial cybersecurity.
524
00:33:15.400 --> 00:33:17.279
So if I can take you sort of on a
525
00:33:17.319 --> 00:33:22.359
side trip real quick here. You know, throughout this interview,
526
00:33:22.720 --> 00:33:28.039
I have been surprised by you personally. I mean I
527
00:33:28.039 --> 00:33:32.680
had always had a stereotype in mind for people who
528
00:33:32.920 --> 00:33:36.200
found vulnerabilities, who hacked stuff, hardware, software or whatever. The
529
00:33:36.880 --> 00:33:39.559
stereotype that I had in mind was sort of somebody
530
00:33:39.559 --> 00:33:42.240
with a big ego, somebody with an ego, you know,
531
00:33:42.519 --> 00:33:45.799
saying to themselves, I'm smarter than you are. I can
532
00:33:45.920 --> 00:33:50.160
find these problems. You know, you the vendor have messed
533
00:33:50.160 --> 00:33:54.119
it up. I always thought you needed that kind of
534
00:33:54.359 --> 00:33:58.640
attitude to be able to go in and tackle you know,
535
00:33:58.720 --> 00:34:02.079
the vendor's defenses in you know, and incorporated in the product.
536
00:34:02.200 --> 00:34:04.319
I always thought that you needed attitude. But you know,
537
00:34:04.720 --> 00:34:07.200
what's coming across from you is something different. Can you
538
00:34:07.240 --> 00:34:11.519
talk about who, you know, what do you need sort
539
00:34:11.559 --> 00:34:14.960
of in your brain, in your personality to be successful here?
540
00:34:15.639 --> 00:34:20.199
Well, to make sure, you just need curiosity and persistence.
541
00:34:20.480 --> 00:34:23.639
I think people with a big ego they're more successful
542
00:34:23.719 --> 00:34:27.320
in finding more vulnerabilities. But like I said earlier, this
543
00:34:27.400 --> 00:34:30.679
is more an expensive hobby for me, so I do
544
00:34:30.800 --> 00:34:35.239
not really have the pressure to find vulnerability after vulnerability
545
00:34:35.360 --> 00:34:38.719
For me, it's more well being on this scavenger hunt
546
00:34:38.719 --> 00:34:42.400
to go away that or find a way to operate
547
00:34:42.440 --> 00:34:45.840
the device it was not intended to and then really
548
00:34:45.880 --> 00:34:48.559
find a way in. And to be honest, I also
549
00:34:48.599 --> 00:34:52.760
have a whole box of scrap OT devices where I
550
00:34:52.800 --> 00:34:56.159
did not find a vulnerability. So this is where we
551
00:34:56.199 --> 00:35:00.800
come back to the expensive hobby. I think if someone
552
00:35:00.920 --> 00:35:05.320
is understanding a bit of the domain these devices are
553
00:35:05.440 --> 00:35:10.039
operated in, and have enough curiosity and then persistence to
554
00:35:10.159 --> 00:35:14.320
stick to it, they can definitely find some vulnerabilities, or
555
00:35:14.559 --> 00:35:17.119
if not well, I can at least learn a lot
556
00:35:17.519 --> 00:35:21.440
about the devices, how they operate, and how they interact
557
00:35:21.480 --> 00:35:23.840
with other devices in the OT domain.
558
00:35:27.119 --> 00:35:33.719
So, Andrew, we've been talking hardware vulnerabilities. It seems relatively serious,
559
00:35:33.840 --> 00:35:37.679
but bring it to a practical level for me, if
560
00:35:37.880 --> 00:35:43.719
I'm running an industrial site and I discover, you know,
561
00:35:43.760 --> 00:35:47.159
a hard coded issue in one of my gateways, am
562
00:35:47.199 --> 00:35:54.039
I running around red alarms, ringing to patch immediately? Or
563
00:35:54.760 --> 00:35:57.880
am I more focused on these systems and data flows
564
00:35:57.920 --> 00:36:02.039
around it that enable sort of legacy technologies to occasionally
565
00:36:02.719 --> 00:36:06.000
have vulnerabilities like this. How would you interpret it in
566
00:36:06.039 --> 00:36:07.639
the grand scheme of things in.
567
00:36:07.599 --> 00:36:09.199
The grand scheme of things, there's there's sort of a
568
00:36:09.199 --> 00:36:11.519
couple of different questions. Let's let's let's pick it apart.
569
00:36:11.559 --> 00:36:15.280
What we've been talking about primarily is how to find
570
00:36:15.400 --> 00:36:19.559
these vulnerabilities. Once you've found a vulnerability, now you've got
571
00:36:19.559 --> 00:36:22.960
to ask the question, you know, a can I patch
572
00:36:23.039 --> 00:36:25.199
the system? Because if it's you know, a vulnerability in
573
00:36:25.239 --> 00:36:28.760
your safety system, well I'm sorry. The testing cost of
574
00:36:28.800 --> 00:36:31.320
the new version is going to be prohibitive. It's just
575
00:36:31.440 --> 00:36:33.719
really hard to patch some things. Other things are easier
576
00:36:33.719 --> 00:36:36.599
to patch. So can I patch it? Second question is
577
00:36:36.760 --> 00:36:40.320
do I need urgently to patch it? And that's sort
578
00:36:40.320 --> 00:36:42.480
of a different skill set. It's one skill set to
579
00:36:42.519 --> 00:36:44.840
find the vulnerability, it's a different skill set to say, well,
580
00:36:45.199 --> 00:36:48.119
how would an attacker? So it's an imagination thing, imagine
581
00:36:48.159 --> 00:36:51.800
how would an attacker use this against me? And you know,
582
00:36:51.880 --> 00:36:54.519
we talked about two scenarios for this vulnerability. One is
583
00:36:54.599 --> 00:36:57.199
physically walking up and stealing the device and taking it
584
00:36:57.239 --> 00:37:00.840
apart and putting it back, which seems not a very
585
00:37:00.920 --> 00:37:04.639
credible threat because you're going to be discovered. The second
586
00:37:04.719 --> 00:37:09.159
scenario was, you know, someone with much more resources discovers
587
00:37:09.199 --> 00:37:12.159
that you've just ordered fifty of these intercepts. The shipment
588
00:37:12.400 --> 00:37:15.079
bribes the driver to go take a long coffee break,
589
00:37:15.559 --> 00:37:17.599
breaks into you know, five or ten or fifteen of
590
00:37:17.639 --> 00:37:23.280
these devices, inserts malware, packages them all up again again,
591
00:37:24.119 --> 00:37:26.400
is that a credible threat. It's a credible threat for
592
00:37:26.639 --> 00:37:31.320
some very high value targets. Is it a credible threat
593
00:37:31.320 --> 00:37:35.400
for a small bakery? You know, probably not. So you know,
594
00:37:35.480 --> 00:37:39.039
first step is fined it. Second step is figure out
595
00:37:39.519 --> 00:37:42.840
can I even patch it? Third step is how would
596
00:37:42.960 --> 00:37:47.079
a bad guy exploit this? Are there credible threats? Is
597
00:37:47.119 --> 00:37:51.679
there a third scenario that we haven't imagined? So it's
598
00:37:52.199 --> 00:37:55.440
a question of imagination and studying what people have done
599
00:37:55.480 --> 00:37:59.960
in the past, and then you know, the decision part
600
00:38:00.119 --> 00:38:02.719
part of it is you know, how easy is this
601
00:38:02.760 --> 00:38:08.039
to exploit? So we're talking about devices generally, we're also
602
00:38:08.079 --> 00:38:11.440
talking about cloud connected devices because a lot of the
603
00:38:11.480 --> 00:38:13.880
devices that Marcelle has focused on that he teaches you
604
00:38:13.880 --> 00:38:17.159
about his industrial Internet devices, they're connected out to the cloud.
605
00:38:17.639 --> 00:38:23.079
So that's more Internet connected, more Internet exposed. But really
606
00:38:23.280 --> 00:38:27.440
what he looked at here was an O T cloud
607
00:38:27.679 --> 00:38:34.559
remote access device. It's arguably the most exposed piece of
608
00:38:34.599 --> 00:38:37.679
technology in the OT network. It's the technology that gives
609
00:38:38.440 --> 00:38:42.920
Internet based users access to the OT system. So normally
610
00:38:43.639 --> 00:38:47.199
you would set these things on automatic update. Why what
611
00:38:47.280 --> 00:38:49.920
if they blue screen, Well, nobody cares if their blue screen.
612
00:38:50.000 --> 00:38:53.199
It's inconvenient if they blue screen. If the bad guys
613
00:38:53.239 --> 00:38:56.039
get in, you know, they can work whatever they want
614
00:38:56.159 --> 00:39:00.639
sabotage on your OT network. So normally people pay a
615
00:39:00.840 --> 00:39:04.519
lot of attention to defects in there. You know, there
616
00:39:04.519 --> 00:39:07.239
are too too vulnerabilities in their O TI remote access.
617
00:39:07.519 --> 00:39:10.119
This one we just you know, we couldn't imagine a
618
00:39:10.199 --> 00:39:14.599
credible attack scenario pre mere mortals. You know, it might
619
00:39:14.639 --> 00:39:18.239
not be that
that big to worry about, but generally speaking,
620
00:39:19.239 --> 00:39:21.679
you know, this is the kind of device you want
621
00:39:21.920 --> 00:39:26.199
people like Marcel picking apart the most thoroughly because this
622
00:39:26.280 --> 00:39:28.840
is the device that has to be the most thoroughly protected.
623
00:39:31.440 --> 00:39:33.159
Before we let you go, can I ask you to
624
00:39:33.239 --> 00:39:35.239
sum up for our listeners, what should we take away
625
00:39:35.280 --> 00:39:38.360
from this? What you know, what's important to know about
626
00:39:38.400 --> 00:39:40.880
this stuff, and how do we use it going forward?
627
00:39:41.400 --> 00:39:44.159
Okay, looking at the vulnerability, I found this was a
628
00:39:44.400 --> 00:39:49.000
prime example that just one part of security was completely overlooked.
629
00:39:49.440 --> 00:39:51.760
When you look at the device from a network perspective,
630
00:39:51.840 --> 00:39:57.000
you see a very fortified device, but security doesn't stop
631
00:39:57.000 --> 00:39:59.880
at the network in our face and also the PC
632
00:40:00.079 --> 00:40:04.599
be the hardware level should be taken into consideration. And
633
00:40:04.639 --> 00:40:08.599
in general, I think OTI security needs more curious minds
634
00:40:08.599 --> 00:40:12.920
that are looking under the hood. For example, if you're
635
00:40:12.920 --> 00:40:17.960
an engineer, you already understand the industrial processes and here
636
00:40:18.039 --> 00:40:21.800
I just can recommend you to level up your cybersecurity skills.
637
00:40:22.840 --> 00:40:26.719
And this is exactly what I'm doing with Foxgrit. This
638
00:40:26.840 --> 00:40:31.639
platform exists to teach industrial security in an affordable and
639
00:40:31.840 --> 00:40:38.119
practical way. The flagship cause Practical Offensive and Practical Offensive
640
00:40:38.440 --> 00:40:43.000
Industrial Security Essentials comes with an open source lab where
641
00:40:43.000 --> 00:40:46.840
you not only learn about penetration testing tools, but also
642
00:40:47.400 --> 00:40:51.639
how you can use them on simulated industrial controllers and
643
00:40:51.719 --> 00:40:55.920
that way you also can understand how your real devices
644
00:40:55.960 --> 00:41:01.559
would behave under such conditions. For the next steps. If
645
00:41:01.559 --> 00:41:05.960
you're curious, check out Fox Sprit for resources and connect
646
00:41:05.960 --> 00:41:09.599
with me on LinkedIn, and of course keep pushing OT
647
00:41:09.800 --> 00:41:10.760
security forward.
648
00:41:14.119 --> 00:41:17.039
So that seems suggest about do it. For your interview,
649
00:41:17.079 --> 00:41:20.559
Andrew with Marcel rick Zen, do you have any final
650
00:41:20.639 --> 00:41:23.280
thoughts that you'd like to share before we leave today.
651
00:41:23.400 --> 00:41:26.960
I guess so. I mean, I had always been curious,
652
00:41:27.079 --> 00:41:31.760
you know, how people do this stuff. What surprised me
653
00:41:31.840 --> 00:41:35.239
about the interview here was that I actually followed what
654
00:41:35.280 --> 00:41:38.280
he did. I kind of understood it. You know, I
655
00:41:38.280 --> 00:41:40.360
thought it'd be harder than that, and I suppose it
656
00:41:40.400 --> 00:41:42.280
could be if you have to, you know, if you
657
00:41:42.280 --> 00:41:44.239
don't have a small amount of information to look at,
658
00:41:44.280 --> 00:41:46.039
if you've got to look at the entire firmware and start,
659
00:41:46.159 --> 00:41:49.840
I don't know, disassembling megabytes of firmware looking for vulnerabilities,
660
00:41:50.880 --> 00:41:55.320
that would strike me as harder. You know, this seemed
661
00:41:55.920 --> 00:42:00.719
really straightforward. You know, I don't know if I don't know,
662
00:42:00.719 --> 00:42:04.000
if I'm curious enough about, you know, how this stuff
663
00:42:04.039 --> 00:42:06.039
works that I would do the work myself. But I
664
00:42:06.079 --> 00:42:08.199
sure wouldn't mind another two or three guests like this
665
00:42:08.320 --> 00:42:10.960
to walk us through how they did the hard work,
666
00:42:11.679 --> 00:42:14.800
so that you know, we can satisfy our curiosity and
667
00:42:14.920 --> 00:42:18.639
beyond my curiosity. You know, I agree with Marcel. We
668
00:42:18.719 --> 00:42:22.840
need people tracking down vulnerabilities. That's you know, it's because
669
00:42:22.840 --> 00:42:27.760
that's the good way to persuade vendors to invest more
670
00:42:27.760 --> 00:42:30.480
in security, to you know, make these devices more secure.
671
00:42:30.519 --> 00:42:32.800
To begin with is to point out afterwards they've got
672
00:42:32.840 --> 00:42:35.280
problems and you know, the next time around, hopefully they
673
00:42:35.320 --> 00:42:38.559
will be more careful. The bad way is to wait
674
00:42:38.599 --> 00:42:40.760
for the bad guys to find the vulnerabilities and exploit
675
00:42:40.760 --> 00:42:42.920
them and take advantage of us. So, you know, we
676
00:42:42.960 --> 00:42:45.199
need more of the good guys. You know, we need
677
00:42:45.239 --> 00:42:49.599
more more technical, curious people out there fight in the fight.
678
00:42:49.760 --> 00:42:51.280
So you know, thank you to Marcel.
679
00:42:52.039 --> 00:42:56.719
Well, thanks to Marcel for satisfying our curiosity. And Andrews
680
00:42:56.760 --> 00:42:58.320
always thank you for speaking to me.
681
00:42:58.920 --> 00:43:00.000
It's always a pleasure. Thank you.
682
00:43:01.039 --> 00:43:05.280
This has been the Industrial Security Podcast from Waterfall. Thanks
683
00:43:05.320 --> 00:43:07.199
to everyone out there listening.