ТРАНСКРИПЦИЯ 🔗
WEBVTT
1
00:00:00.000 --> 00:00:13.000
The best case scenario is I turn it on, I'll let the people in do the work and then I turn it back off. Like a work order. It should have a review date. There should be approval. Who can approves. It should be in your change management documented. Like everything should be documented.
2
00:00:13.000 --> 00:00:32.000
You're listening to Protect it all where Aaron Crowe expands the conversation beyond just OT delving into the interconnected worlds of IT and OT cybersecurity. Get ready for essential strategies and insights. Here's your host, Aaron Crow.
3
00:00:32.000 --> 00:20:00.000
Hey everyone. Welcome to another episode of the Protect it all podcast. This time I want to talk through all things connectivity in the OT space. One of the, the. I don't know, one of, one of the main issues that we, we normally see struggling with obviously outside of asset inventory visibility is remote connection. And because we have so many older systems, unpatched systems, we don't necessarily know all the systems vendors hands are in things and lots of connections and backdoor. Backdoor is probably not the right word but you know, alternate paths into and out of our environments. Controlling access is vitally important to these environments. Obviously a lot. There's a lot of expertise in engineering that goes into these spaces, designing these control Systems and, and PLCs and, and all of these very, very specific environments. So we really many times are leaning upon our vendors to be able to help us to support, to provide that level of expertise when we're having issues, all of that type of stuff. Right. So that, that leans heavily on really being able to protect that. So you know, if you, if you had to look at this CSF or you know, the control, control areas, one of the factors or the stronger areas that, that most organizations need to really focus on and be strong on is that secure remote access. It's that connectivity to these spaces. How can I, how do I allow people to access? And that doesn't necessarily mean remote being, you know, at another, you know, physical location. It could be, you know, even just how my technicians are accessing these networks. Are they bringing in transient devices? Are they plugging in their laptops? Is there a dedicated device? Are they connecting from their office that's across the street? Are they able to connect from the corporate headquarters that's down the road in another city? I connect from another country. Do I have third party vendors that are connecting in all of these things are factors that cause impacts or could cause impacts to these environments. And beyond just cyber issues, there's many of these issues. And this goes back to. I need to understand what's happening in my Environment like when we walk into a place and, and something stops working. I literally just got out of my son's truck and that it wouldn't start. It's an older vehicle and it wouldn't start. So what's the last thing that's changed? That's, that's what I'm looking for. Who made a change? What's the biggest impact that could have caused this thing? And let's start reverse engineering where that did that get to so that I could start saying hey, if, if he just installed new lights on his truck, maybe he forgot to connect the battery or anything like that. Right. So it's really just being able to look back at those things. So specifically today I want to talk about CISA and the UK National Cyber Security center and along with another a bunch of other different entities have jointly published practical guidance on secure connectivity principles for ot. They offer an eight point framework to design secure manage connectivity and OT environments as organizations face raising business blah blah, blah, blah blah. Right. So what does that mean? Like they're really giving guidance because obviously someone else agrees and believes that this is an issue and a risk and something that people need guidance on. Because we know that the days of 20 years ago when I started having these conversations in these OT spaces that it was always just like oh well, we'll just disconnect. We're not going to allow remote access into our environment. You've got to be on site to be able to touch that. I think that has changed obviously over the years. I know GE is very large. They have their Atlanta data highway as part of their control system and they did a lot of monitoring across that network which was value add to the customers. But that also meant that they were able to access that from the country across the world depending on where, where your facility was located. And yes, you want GE doing that. You don't want a bad actor being able to do that or even a you know, disgruntled employee or even just something accidentally happening. Right. So what does that mean? So the eight sections, it's really around just like we I harp on and I even started this conversation with the design connectivity for documented business seed. What does that mean? Right. I really need to understand my environment and make sure that I'm not just doing an any any. I'm not just allowing access from anywhere to anywhere at all times for all users. It sounds obvious but many times when we're walking in these places they have something that was set up five years ago. They don't exactly know why it's still there. They're hesitant to turn it off because they don't know what's going to break. The problem with that is, is that anybody could be getting in. And add on to the fact of that that we may or may not be monitoring those environments or that that network connection or that that device, then that adds more problems. Right? So you know, you want to, by, by designing connectivity and reducing that, you know, you're reducing your unnecessary attack surface. I'm not disallowing RDP from anywhere on my corporate network or from anywhere, even just within a certain subset. It also limits who is able to
Load full transcript
get in. Like even when I was an asset owner and I was the one responsible for OT in those spaces, my team would call the local site before we logged in and made changes. And the way my, my leadership explained it. Scott Rosenberger, if you're out there and you're listening, you know, you don't walk into somebody, at least not in Texas, you don't walk into somebody's house and start eating out of their fridge without letting them know you're coming over. That, that would, that would give you a probably bad experience, especially in, in Texas. So, you know, reducing unnecessary tax surface and, and not allowing people in, you're not making changes that nobody's aware of. I don't know what changes are going on at the site. So I don't want to be rebooting something when you may be in the middle of an outage, you may be in the middle of troubleshooting something. I just rebooted your system. It sounds innocent enough, but that, that happens and that can cause problems. Right? So all of those, all of those connections need to have a valid documented business justification who is owning it, who's responsible for it, why is it there? Is it open all the time? You know, when does it turn off, when is it, it should time out, it should expire. I shouldn't just have this RDP connection open all the time. The best case scenario is I turn it on, I'll let the people in do the work and then I turn it back off. Like a work order. It should have a review date. There should be approval who can approves it. You know, should be in your change management documented. Like everything should be documented. Right. But that all leads down to. You need to build and maintain a definitive OT inventory. You know, before you enable access. You want to understand what they're getting to. Because if you don't have a valid inventory and I'm giving access to system A and I Don't really understand my network. And what other systems are there? Giving access to system A could mean that I'm also giving them access to system B, C, D, E and F, which could cause problems. Obviously pivoting they should. They could be on a system they're not aware or meaning to. They could accidentally connect to things. Like there's just all sorts of things. Right. So you really need to secure or let me rephrase that. You can't, I hate to say can't. I know Dale Peterson doesn't like this. You can't secure what you can't see. You can secure things you can't see, but it makes it really hard. It makes it much harder to secure and accurately, effectively secure an environment that I don't know what's on it. Yes. If I just turn everything off and I don't allow anything in and I'm not, no traffic is coming in like in a nuclear environment that's completely really only air gapped place that unless they have like a data diode, like a waterfall, there's no data coming in and out. Yeah. I don't necessarily have to know what's behind the fence. But most OT environments are not like that. Right. They, they are connected, they are allowing remote access, they are allowing PI data out, they're allowing, you know, they're patching, they're, they're getting data from other places and you, they're updating systems and servers and databases and all of those things are inherent risks. So you really need to understand them. Right. So a continually updated record of assets, firmware versions, flows, ownership roles and responsibilities, support contracts like those are all. They sound so simple and it's not sexy and it's not AI but it is the thing. Probably one of the biggest things that people can do is manage and understand their environment, their assets and their network sections. Right. You want to minimize and isolate connections. I, I'm a very, very strong believer in segmentation and least privilege. You need to isolate environments as much as you can. My, you know, system a microsystem, you know, my, my turbo control system doesn't necessarily need to be connected to my balance of plant. Like maybe there are some connections need to be there, but they need to be intentional. I shouldn't just automatically connect everything on, on network A to network B. And again, this is not, this is just good network architecture. This is just good system design. Even beyond just cyber and risk of a cyber attack and ransomware and, and all of that. It's just a good idea. I've got many stories and I've told this story. I don't know if I told it on the podcast, but you know, prime example of a, a site where they were doing a control system upgrade on one unit and unit, Unit one and common were online. Unit two was offline. They were upgrading unit two, but it had active directory and it was all one big shared network. Long story short, the upgrades that the vendor did to Unit two impacted unit one to the point that the operators had to punch the unit out because all of their, the data on their screens in the control room went blank. They had just smurf data, nothing, right? They couldn't, the plant was running fine, but they couldn't see it or control it in their feet, in their fear or their theory or what they could see. So they turned it off until figured out what was going on, right? So there's a prime example, had nothing to do with cyber security. It wasn't a nation state, it wasn't a bad actor. There was no ransomware, you know, none of that. It was just. It impacted operations, right? And for a plant manager or for a site, they don't care. It doesn't necessarily matter if it was China or if it was, you know, some third party country or, or, or some bad actor. Did it impact my environment? Right? Was it an OT attack? It doesn't matter. Did it impact, you know, VLANs are good. Do you really want, you know, ACLs? Yeah, sure. FireW are better. Zero trust. Micro segmentation is, is, is best in some scenarios. But you know, you really need to understand when you're building out your asset inventory, you need to understand the things that are critical, what their job
and their role function is, so that I can make sure that on those critical assets, I'm understanding them, right? Colonial Pipeline. You need to understand where your OT assets are, which ones are critical and which ones can go down or, or have an issue. You need to have authentication and authorize all of those connections. You want to require strong multifactor authentication, you know, dual factor yubikeys or tokens and you know, whatever your preferred method is. You want to have some kind of MFA for accessing these environments, right? You want to take away the, you know, the vulnerability of passwords being your only protection in these spaces. Especially because most people are going to use the same password in their home life as they are on here just because they have so many passwords to remember. And it. If you're not using a password manager, which you all should be, then it makes it really Hard to remember all those passwords. So, you know, you're going to prevent credential use, simple brute force attacks, you know, lateral pivot attacks. Just because I have access on this system doesn't mean I have access on that one. You know, certificates and, and session tokens, jump servers. You want to record sessions, recording sessions get you a lot of things, but, you know, some level of, of, of understanding what happened in my space. I can't tell you how many times a vendor would log in, make a change, something changed, we weren't sure what we call the vendor. I didn't make any change. And then we go back to the logs and say, yeah, who made this change? Right here. And we found out, hey, that was Bob. Bob made this change. Why did Bob make this change? May not be a bad thing, maybe a good thing. He just didn't tell us or update the work order, whatever that may be. You want constrained and direction limited communications, right? I, I, you not usually in, in most of my architectures, I'm not going to allow somebody from outside my corporate network to initiate a connection directly all the way to the OT environment, right. I'm going to have some level of relay in between there. They're going to have to authenticate into my network, multi factor VPN in. Then I'm going to hit them with a jump server that's going to then authenticate through different credentials from the outside and the inside, you know, some level of, of, you know, delegation of duties in those spaces. But I definitely don't want to allow something way out to come in. A lot of these newer really amazing secure mode access solutions enable a lot of these things from having the password management to having, you know, credentials and certificates and recording sessions. And a lot of these things, not to mention that they're, they're, they're splitting protocols, meaning they're, they're not using, it's not RDP from the Internet. You're not connecting your laptop directly to it. It's a protocol break. So it's, you know, HTTPs on the outside. So you're, you're accessing it from a web page and then it's doing RDP directly from the back end. But you don't have the ability to copy and paste files directly like onto the desktop, so it's harder to pivot. I've got to go through a system which it's actually scanning the, the files to make sure they're not malicious. It doesn't mean it's going to find it, but because it's a Whitelist, blacklist. But the point is, is it's making all of these things harder. The struggle that I've seen in a lot of these spaces is because it's also harder for the, for the good guys, right? It's adding a step, it's adding a little bit of complexity. So we have to make sure that we're not over engineering it so hard that the, the, the normal day to day actions are so difficult that I can't, you know, I can't do my job. The engineers can't do their job, the operators can't do their job because then they'll just find workarounds and we don't want that thing either. So you know, you want constrained direction, limited communications, but it has to be able to work. We talked a bit about it, but you want to make sure that you're, you're doing visibility, logging, telemetry, session recording, you know, those are all super important things for obvious reasons. The session recording is huge. Like it's, again, if nothing else, it can even help with documentation. Hey, I've got this vendor coming in to make these changes and update the system. I can document all the steps that he took so that the next time this happens. I've got a documented process on how to do those things and to update my work orders, update Maximo or whatever my system is. All those things are great. You know, you want to make sure that your provisioning and vendor channels have, you know, firmware and you're updating your firmware in a right way. Like you're not just randomly going most, most, many, I won't say most, many of your vars and vendors are going to provide you a secure place to be able to go get your firmware, right? They're not just going to drop it. You know, you're not gonna have to just go to the random website and get it. Like they've got hashes that you can confirm and go through. You know, all of this is to make sure that your environment stays your environment and it's not impacted. The bad thing is, is if somebody gets in, it's really hard to get them out. It's hard to see that they're there and it's hard to get them out before they do bad things. There's, there's multiple scenarios where you know, there was no secure mode access. Somebody brings in a, a transient device about, you know, their own laptop, whatever, plug it in. Now they've got a connection going out because it's really easy to mon. Initiating from outside in and block things. Many times I've seen firewall rules where they don't allow anything in, but they allow almost anything out. And that's where those communications can be established from inside, reaching out. So we need to make sure that we're monitoring those things. So all this to say it's, it's a good process. You know, they're, they're, they're digging into, you know, limiting exposure connectivity, centralizing network connections, you know, using standardized and secure protocols, you know, harden your OT boundary, you know, limit the impact of your compromise. Like all of these things are things that we talk about all the time. So it's not, it's not rocket science what they're talking about here. But it also goes to show that these are basic principles that there's still so many that are not, have not implemented yet, at least not consistently. Right. So how do you guys feel about these types of things coming out from government entities? Do you think it'll help, does it help you with your conversations? I can see this being a good thing to come across to be able to rise up to, to justify, scope, resources, etc to really help to push these narratives through, to get budgets aligned and assigned, you know, to justify spending and resources and all those things. But my fear is, is that a piece of paper or something like this alone may not be enough. Right. Does there, you know, in the power utility world and on the, on the US side, NERC SIP has come along and yes, there's a stick, there's a, there's a fine, there's a, there's an impact, a negative impact, financial negative impact if things aren't done. So does it need to have that? Some will say yes, some will say no. I can argue both sides of that coin. I think there's a place for it. I'm also, I don't love compliance because then I think it's very easy to miss the, the force for the trees that I'm compliant, but I'm not secure. It's, it's a slippery slope for sure. But I definitely believe that we have to do more and do something to be able to push this through. So love to hear what you guys think. If y' all think this is a good thing, if you disagree, you feel like, if you like the wording, if you think it's too vague, if you don't like that it's so pointed like what are your thoughts and feelings on it? Make sure that you like subscribe all the things leave comments Always looking for folks topics like this one. I appreciate Perry for sending me this this to have a conversation around. Definitely an interesting one and definitely one that I'd love to see what the response from the industry is and how this actually helps or hinders depending on how people look at it from from this point. So thanks a lot.
4
00:20:00.000 --> 00:20:26.000
Until next time, thanks for joining us on Protect it all, where we explore the crossroads of IT and OT cyber security. Remember to subscribe wherever you get your your podcasts to stay ahead in this ever evolving field. Until next time.