Pełna transkrypcja odcinka Episode 109: AI-Powered Security Threats

was too much to sustain. They didn't have a system for managing that complexity, didn't manage their defenses, and eventually everything fell apart, and fell into something that could be managed because each of those units was less complex and could be managed by the systems they had at the time. Fascinating idea. Where am I going with this [laughter]? TIM: Yeah, where are you going with this [laughs]? MIKE: The barbarians are at the gates [chuckles]. We have a rise of AI now, and the tools that are available to exploit vulnerabilities are now potentially world-class, available to not just nation-states, but...you know, and maybe not quite the script kiddies, you know, somebody in the basement thinks, "Hey, what can I do here?" But getting close to that level. Some of the main things that are preventing it right now are cost. And some of the large frontier models are, you know, big companies publishing frontier models are holding back the strongest functionality for vulnerability detection because they know they can cause so much trouble. There are a couple of projects out there, Project Glasswing from Anthropic and Project Daybreak from OpenAI, where they're partnering with large government and business entities to try to test their systems, test libraries, and get things hardened because the clock is ticking. In some ways, we've already reached the point where it's starting to tick up. We're already seeing news items all over the place. If you look at your logs in your servers, you're probably seeing attacks. You've always seen the kinda script kiddie kind of attacks. Low-hanging fruit have been out there for a long time, but it's ramping up, and I'd like to talk today about what we can do about it. I've got some thoughts, but with...we've got Justin Ellis here. He's a long-standing security engineer who has worked in Acima in the past, but is not currently. Tim comes from our security team. Will has worked for a variety of large corporations [laughs] and can speak from that perspective. And, you know, so I think we've got a good crew here. And Vivian's recently come out of school, so she can talk about [laughs] things from, you know, the perspective of somebody coming to the industry today, because it's very different than the industry looked five years ago. You know, we're in a different world. I do have some thoughts on here about some tactics we should be doing, but I'd like to stop talking myself and get some initial thoughts from y'all. JUSTIN: So, just one thing: I'd like to, like, share the current status, and then, you know, other people can share. It used to be where somebody would crack a network or crack an organization, and then they would sit in that organization, you know, hidden inside their network, trying to scout out other things that are vulnerable inside and slowly go through that and spider web, and then do something with it, whether it's a ransomware, or something like that. But it was, like, the average time that they were in that network was, like, measured in, like, 150 days, so it was measured in, like, 100-plus days. Now, if your outer defenses get cracked, you can lose your inner devices almost instantaneously because of the AI assistance that is part of that attack. And they can go out and, you know, crack all of the vulnerable devices inside your network very, very rapidly and then decide to do something with it almost autonomously. I mean, that's the whole point to the AI, right? And it's making it such that the, you know, crack to ransomware or, you know, compromise of your entire system is measured in minutes rather than, you know, days, where it used to be, you know, hundreds of days. And so, it's just a very different landscape for cybersecurity than it used to. So, I'll just leave that there to start off with. TIM: Yeah, we had some...I recently was working with Mandiant on an incident, and some things that they said which were interesting to me was that the median time for groups like Verizon or big telecommunications groups to patch a zero-day or something that, like a CVE 10, 9, somewhere in there, something that's pretty critical, is about 43 days. And with the advent of AI, the average time from exploit PoC publication on a darknet or a dark web community to demonstration of that exploit being exercised is about five days now. So, someone will find a zero-day, put it on their forum, and then people with enough GPUs to run Kimi 3 on their local rig at home will then use that PoC to then roll out a campaign against, you know, against whatever their intended target is. And they'll pop a target faster than that target will ever be able to roll out a rolling update or some kind of change to fortify against that. So, at this point, every security engineer right now is like, "Shit," you know [laughter]? JUSTIN: Yeah. You get me [laughter]. TIM: Like, yeah, it's so hard because, like, I try to explain to my leaders and everyone like, "Hey, we need to get...we've been trying to upgrade the standard that we have as a business. When we identify something that's critical, we need to get it done within, like, a business week. That's the target we should be aiming for." Whereas the standard up to date has been, like, 30 days, somewhere in there. And that's really hard to do between application development and then, like, your server-side administrators that are, you know, protecting, you know, an Oracle whatever server on an old rack in the corner of a data center. You know, it's a lot to take in. WILL: Well, I mean, I'm curious, right? So, like, a couple of iterations past in my career, like, I was doing a lot of sort of, like, validation work on, like, a OS level, like, really deep nitty-gritty, like, on the sort of, like, the surface layer where these exploits would pop up. Like, where you're exploiting, like, you know, the things that were written, and they were written, generally speaking, like, pretty well, but, like,

real nitty-gritty threading, memory management, like, ways of breaking the...I mean, effectively you're breaking complicated programs, right, in predictable and exploitable ways. And I can see very clearly how you could take AI, you know, frontier model tools to, you know, effectively write, like, a next-generation fuzzer, right, to interface with these APIs in, like, complicated ways to try and break them. Like, that is a pretty clear A-to-B sort of a thing. But creating a patch for those things is breathtakingly hard, well outside the capability of most models. I mean, because this is stuff that I have some familiarity with and, like, you can't just...like, not everybody is just going to do that stuff. Writing a kernel patch is well beyond the capacity of anybody who...like, very few developers could do that. But now we're, I think, we're faced at a situation where, like, the average, let's say, North Korean [laughs], you know, North Korean exploit writer could create these exploits relatively efficiently, let's say. You know, if you say like, "I've got a guy full-time, you know, I'm going to get him a bunch of GPUs. If he gets one home run a year, that's going to pay for this whole thing." Whereas, like, on the other side, it's just, like, you know, how many developers? Like, I don't know if you've got a person who could do that work at Acima, like, you know. TIM: No one. WILL: No one. TIM: I don't know anyone at, like, a big shop that...we don't. No one has that capacity. Man, it's just...So, Chance, who's my peer, so I do the cloud security side of things, and Chance does the application security side of things. And we talked about this after going to DEF CON last summer. Because we were participating, planning, and doing, like, these capture-the-flags, and I went there with the intention to...my little hobby was to hack my hot tub. They have, like, an IoT, whatever they call them. They're, like, communities, but they're, like, little hacker huddle kinda things. And I like to go to the IoT community. And I brought in my little hot tub controller, and we sniffed it and kinda reverse-engineered its API, and that was a lot of fun. But what has changed dramatically, which we noticed last summer, was these CTFs, these people who are competing, no one was pulling out, you know, the normal tools and doing the kind of the recon they usually do. They were spinning up local models on their machines or tapping into some external resource. And they were letting the models of their choice do all the busy work. And it almost took the fun out of it. Like, I remember Chance was like, "Dude, this is really boring. Like, this isn't fun anymore [laughter]. Where's, like, the edge of competition?" You know, even, oh, what's that company that makes electric cars? Not Tesla, the one that made that truck. JUSTIN: Rivian? TIM: Yes, Rivian. They brought in a couple of their cars to DEF CON and said, "Go for it. Like, please try and hack into this

thing." And it was really cool to see, you know, a dozen teams tearing apart a Rivian and pulling out the cards and everything, and they figure...but they were all doing this with augmented AI assistance with everything they did. And after we came back from that, we talked to David, our SVP of cybersecurity. We're like, "It's changed now. We no longer can rely on our ability and our experience to effectively defend an organization. We have to be just as tied at the hip to AI as the attackers are. Otherwise, we will never repel a defense, you know, offer some kind of defense for our organization at a commensurate rate at which we are being attacked." And this year at Black Hat, we're seeing that cybersecurity vendors are finally catching on to that. They're embedding agents into their tools. Tools like Orca and Wiz now have red team, blue team, green team agents. They have AI-assisted tools that are designed to do what would take us, you know, weeks to get done. It's also changing the way CISA does things. Like, the annual pen test that everyone's required to do, it kinda doesn't mean anything anymore now. You do, like, a pen test once a year, and you're like, "Cool, you patched that. Did you know that there are now 10 more that you didn't even check in your estate of, you know, portfolio?" This side of the industry is changing a lot. I think even just as much as the worry of a, you know, some senior engineer is given an AI plop of slop that he needs to run through a PR and see what was crap and what isn't crap, as much as that's changing how we form software, we're also seeing a massive change in how we keep it protected. It's a lot [laughs]. WILL: I mean, with the IoT thing, I don't know. I mean, like, decompilation, right? Decompilation used to be hard. TIM: No, yeah [laughs]. WILL: And it's not. It's just, like, oh, I want...like, I'm sorry. This is...I, you know, I should move through this, like, moment quickly as, like, an old dev, but, like, there was so much...there was so much around, you know, a library versus your source code, and your source code was protected. Your source code was this precious thing, you know, that you wouldn't let out, and now that's gone, man. Like, an LLM is, you know, really good at...you know, and again, it's...we talked about this before, like, LLMs are good at things and bad at things and translation, right? Translation is something they're breathtakingly capable at, and especially translation of something generated by another machine, and is very predictable and easy to understand ways. And so, God, it's so crazy. I mean, like, oh, I need a device driver for this video card, but they didn't write one for, you know, macOS this version. It's like, oh no, just pull it down. Write your own [laughter]. TIM: Being a little OPSEC-aware, we had an incident just like that three or four months ago where a mobile app that we were responsible for, we had reason to believe that it had compromising information in it, and we were like, "Okay, we have private models that we use in cybersecurity." We said, "Okay, go for it. I want you to rip this mobile app apart and find it." And within 5, 10 minutes, both Chance and I are, "Oh, there it is, like, we found it." And, like, holy...that would've taken me, like, a day. That would've taken me so long to decompile that and rip it out, but now it just blows right through it. It's staggering how uncomforting that is [laughs]. MIKE: Yes, and, you know, we're talking about our own context here, but this is everybody. This isn't Acima. This isn't name your company. Wherever you're working, it's you too. It's all of us. The world has changed, and it's going to change a lot more in the very near future because if you look at the curves of how these things are getting better, it's exponential. Even if it was linear, we can't keep up. We're already behind as an industry, right? We're already behind, and it's going to get worse quickly over the next, like...we're not talking about years. We're talking about months. Maybe even weeks. You know, we're talking about a short time horizon here. It does take a while for people to get familiar with the new tools, and there's costs and, you know, there is...it's harder, right? But you can download some of these big Chinese models, run them on your local rig, like you say, get the right harness and the right target, you're good to go today. It's really scary. I'm not on the security team, right? I'm on the engineering side, and I think about this all the time because this is just central to what we're going to have to do to survive over the coming...it's hard to name your time horizon exactly, but I'll be surprised if, you know, we can go a year at status quo. And, again, when I say we, everybody. JUSTIN: So, talking a little bit about the industry I'm in right now, which is the OT industry, it used to be that a lot of companies would roll their own network security. And they did all the right stuff, like the whitelists and the filtering and such. But now you almost have to have a subscription to a big network security firm in the OT space, like Dragos, or something like that, to be able to keep up with the attacks that are out there. And the comparison on your side would be Wiz Subscription, where Wiz is monitoring every single, you know, node in your network, in your cloud infrastructure, and it is alerting or blocking any abnormal behavior. And you need that type of software that can understand normal behavior and that is authorized to block abnormal behavior the moment it's detected. So, it needs to be able to, like, make that decision without having to wake somebody up at night or anything like that. It just needs to do it, to be able to react instantly when it detects something that it suspects. And then you can come in and...in the morning, you can go in and see, okay, this stuff got blocked. It may have broken something, but at least we know, you know,

we haven't been popped or, you know, something hasn't gone horizontal in the network. MIKE: To your point before, if you have to wake somebody up, you're already too late. JUSTIN: You're already too late. So, that's one of the things, like, I was chatting with a bunch of...I presented this last week to a bunch of people who are financing energy sites. And these are the guys that are deciding whether or not to give money to sites that are asking for money to build new sites. So, I presented to them and told them, "Okay, you guys aren't cybersecurity experts, but here are the questions that you should ask the site operators before you give them money to make sure that they have a plan to address, you know, this new landscape that we're in." And the questions were basically: Do you have a network security software that is one of the big, you know, contenders in this space that is going to be turned on the moment you, you know, the moment you're building this? The next one is: Do you have normal traffic analysis and things like that, or monitoring that you can detect abnormal behavior? And then the last one is: Do you have a disaster recovery plan, and have you practiced it? Because even with all the preparation that you can do and all the defenses you can do, you need to have a plan for when you are popped. TIM: The other one I see, too, is, just maybe a month ago, the U.S. started opening up cybersecurity contractors to act on behalf of the state. So, now you can technically work beside cybersecurity command and the NSA and others to do offensive attacks against other nation-states on behalf of the U.S. And, like, you have to have a...I think it was, like, a million-dollar bail for your own self before you can work with, you know, the government and stuff. But it, like, the cybersecurity side of everything, you know, when you engage someone in your supply chain, you almost have to insist that they have a cybersecurity insurance policy for when they get popped so that your...if you get popped, how am I covered? You know, the intellectual property that you are privy to or storing in your systems, how am I going to be covered from that as well? And there's, like, this whole new industry now of cybersecurity tools, insurance, even some insurance companies that are working on behalf of the threat actors to parlay at ransomware conversations. It's insane, like, how much money is coming out of that whole industry. WILL: I mean, I've long been of the mindset that, how do I put it? Like, IT, let's call it, as an industry, is long, long, long, long, long, long, one more long just in case, long overdue for any form of professional licensure or oversight or anything. I mean, right now, like, how much of our IT infrastructure is living in nations with, let's say, tenuous rule of law, money laundering, you know what I mean? Just sort of, like, security apparatuses. And I'm not saying anything about the quality of the developers, but I'm saying, like,

where are they? It's like, okay, we've got all these people...we have people who are developers, good developers doing great work in active war zones, like [chuckles], you know what I mean? And it's just like, that's...there's an exposure there that the industry, I think, has been, you know, for profit and motive reasons, very hesitant to really investigate at all because, like, you know, like, offshoring just wherever you can find a cheap keyboard. But this is a thing, guys. Like, just anybody...anybody could do it. That guy said he'd do it for $5 an hour cheaper, so let's roll [laughs]. TIM: That's a really good point. This came out today. The story was that as the U.S. was approaching against Iran in spring this year, and just before they were really getting to the throes of this war that we're in, a CIA analyst was presented a finding to leadership that there was a Chinese vessel that was transporting materials for nuclear weapons, and they were in the area of Iran. And that was, like, borderline World War III, you know, something that the military leader's like, "Yeah, we can't have that. We're going to do everything we can to shut that down." And at the very last second, they did their due diligence and looked at, "Okay, where's your information? Where's your source?" Everything like that. And they dug into it. Like, the analyst had actually been using AI to analyze some imagery or something like that, and they were completely off. Like, they didn't...the article doesn't say what it was or anything like that. They kind of keep it a little capped in, but it adds light to this conversation that, yeah, you probably should be qualified before you make really big decisions off tools that you don't know a lot about. That kind of stuff could go wrong really bad [laughs]. MIKE: Well, you bring up an interesting one. Already, I was just reading about this in the last couple of days, at least twice, worldwide conflagration, like thermonuclear war destroying all of human civilization, has been avoided by one person who knew what they were doing saying, "Oh, you know what? This device is telling me that missiles are coming in, and I don't believe it." TIM: Yeah. War games, right? MIKE: What's that? [laughter] Yeah, it comes down to one person, I think, both times in the Soviet Union somewhere, who said, "You know what? It says that I'm supposed to launch the counterattack, but I don't trust this device because I know better." And if they had been, you know, trigger-happy, we wouldn't be talking here. That's scary [laughs]. TIM: Especially when... MIKE: There's risks out there, real risks. TIM: Yeah, especially when leaders talk about moving decision-making processes to automation, and they feel that pressure to do it because the only way they can react quickly enough is to let automation take over for it. But man, that's...yeah, like you said, if you don't have someone that has experience, I don't think that's a good idea [laughter].

WILL: I mean, like, no, I disagree with that. I disagree with that entirely. I mean, I wholesale reject that. It's a question of like, they don't want to spend the money. If we were operating, let's say, a diamond exchange, right, that was open for business 24/7, right? 24 hours a day, seven days a week, like our web servers are, we would have an armed guard on premises, 24 hours a day, 7 days a week, you know what I mean? Like, do you have a person who is, you know, monitoring your servers 24 hours a day, 7 days a week? No, you don't. You go to bed because you don't want to hire somebody to be your night watchman, right? Like, somebody with that skill set to be your night watchman. You don't want to hire that guy. And we've gotten accustomed to a level of laxity that I think is maybe unrealistic and, like, slightly entitled, you know? It's like, no, if we were running a bank, you know what I mean, you'd have a security guard while the bank was open, right? It's like, "No, no, no. We've got one guy. We've got a guy. We have security guard. We have guard [laughter]. Guard works, but we can't...we have to let him sleep, so I guess we just won't have guard then." MIKE: You know, that's maybe a good pivot. This has come up several times in the conversation already. So, we've talked about the risk we're in. I think we've clearly established we're in a different time, and the risk is elevated, and it's just going to get worse. So, what do we do? Going back to, you know, pre-modern times, you got your Roman Empire in ye olden days of yore. You had somebody sitting on the hill, right? Or on the tower. You have that watch person, right? You've got the watcher. In fact, if you've got a large empire, you probably have watchers. So, you've got your Great Wall in China, right? And you've got watchtowers, you know, within sight distance of each other, with somebody staffing each of them because you need to be vigilant. I thought of this when Justin was talking about, you know, your automated system to catch when things happen. If you're not watching, you don't know. And once the invaders are inside the walls, they're already through your defenses [chuckles]. You're in a bad way. So, it seems like, and I'm going to put this out there...I want feedback. Step one: observation. And it doesn't matter how good your walls are, you better be watching. You know, you talk about that bank, Will. Yeah, you've got a bank, and it's probably really well-defended, and it's got a really nice vault. And it's got lots of, you know, it's going to be difficult to guess the combination on that vault. Somebody walking in is not going to find it. But that guard's there anyway. That guard is there anyway, because they don't trust that the walls are sufficient. So, I'm going to suggest that the watchers are critical in a way maybe they never have been before. If they weren't before, they sure are now. Thoughts? WILL: Well, I mean, I'll tell you another thing why I think we need to do away with, I think...and this needs to be done away with fairly permanently. And it's already happened, but it needs to...we need to finish the job. I feel like a lot of our really critical infrastructure layers, right, OS layer, device driver layer, et cetera, are operating on 'trust me, bro' security, right? And I don't think...I mean, we've all seen, like, the enterprise runs on Linux, really, right? But, like, you know, Windows, macOS, stuff like that, like, they're massive security surface layers that are operating on, like, 'trust me, bro' security. "You see this black turtleneck? Black turtleneck would never compromise your data." "But, well, can I audit that? Can I verify that?" "Trust me, bro. We're good." And we know for a fact that many of the browser exploits are not getting fixed on time, you know? Like, we're finding them, and they're just sitting. And I don't think we could afford to operate like that anymore, especially since...you know what I mean? We have these, like, service layers. You can take, like, a Windows surface layer. I haven't been on Windows for a very long time, but I know historically the real serious attack vector for, like, a Windows workstation was through the device drivers, because you'd have some company wrote a third-party device driver that was operating at the kernel level. And they didn't have the resources of Microsoft to keep up with things, but it's still running as root. And so, you crack that device driver from, I don't know, the headphone [laughter], you know what I mean, like, your headphone amp that you got from the cheapest supplier on Amazon, and you're off to the races. Obviously, that's, like, I think, a little bit of a contrived example. But we'll definitely go for the lowest bidder for a keyboard or something, and we have to get a little bit more serious about this stuff than we have been. MIKE: Just to do some data backing that up, Anthropic came out with their Mythos model a couple of months ago. And when they published the initial stats, they had done some vulnerability testing, and they were able to find significant vulnerabilities in every major browser and every major operating system, including things like [inaudible 32:25] security operating system, the BSD. It was one of the BSD distributions. Like, things that have been hardened for decades, they found vulnerabilities in. It's out there. Those vulnerabilities are there. TIM: Just to add on a little bit to that, that was what, four or five months ago now? The open-source models that are coming out, there was a recent analysis done, I think, last week, saying that it used to be that the open-source models were six months behind the frontier models, but now they are four and a half months behind the frontier models. So, it's like, you know, the stuff that Mythos was doing back in the day, back in the olden day of six months ago [laughter], you can have that same capability running on your pretty well-specced-out

MacBook Pro, and you don't even have to be doing it maliciously. You could be saying something to the effect of, "Hey, my model, I give you control over my laptop to go plan me a really cool vacation and make all the things," and your laptop decides to go take over a power site because it wants you to have a good vacation [laughter]. And that sort of thing is going to be entirely possible. I say that kind of joking now, but think of what happens five years from now. MIKE: I read an anecdote just in the last week or two about somebody who was using one of those agents that went and hacked the travel website because the tickets weren't available so that it could get them the tickets [laughs]. TIM: So, it could get them the tickets. MIKE: Yes. JUSTin: So, it's going to be interesting because, basically, I think, every single business out there, every single mom-and-pop is going to have to have a subscription to Wiz that is monitoring their network and that has the ability to shut penetration attempts down. And it's just going to be a business expense that everybody's going to have to pay, and Wiz is going to make gajillions of dollars of money. But that is the thing, because they aren't going to be able to keep up on the internal defenses, all their devices that are running inside, there's no way they can keep all of those individual devices up to date, like IoT stuff, everything. So, you really have to have a really strong outer defense, and that is not even going to be sufficient long-term because, you know, what if that one gets popped? But at the same time, that's almost, like, going to be a for-sure expense that you have to have, that you have to spend. WILL: Like, I almost wonder, like, just the scope of, like, the vulnerability and, like, the difference in sort of regulatory regimes across countries, whether, like, the internet is just not going to be for everyone anymore, you know? Where it's just like, "Hey, yeah, North Korea, we're not going to talk with you anymore. You guys don't get the internet anymore." And, obviously, that's a little bit more complicated than just saying, like, "We're going to kick the plug out," right? Because, like, the Chinese will still...The Chinese and the Russians will still talk to North Korea. And if we still talk with the Chinese and the Russians, then we're talking to North Korea by proxy. But, like, I wonder, to a large degree, whether with the political, legal, regulatory framework being what it is, whether, like, the internet as a whole can truly be worldwide anymore. TIM: Maybe. VIVIAN: I'm going to push back on that actually. There is a huge market nowadays for essentially residential proxies. Every time you buy a projector, or a Roku Stick, or an Amazon Fire TV Stick, or whatever, part of what they are providing is they sell your internet as a proxy to other people so that they can use what appears to be a trusted IP address to do any form of malicious activity or very legitimate activity. But regardless, that is a huge component of the sale of TVs nowadays. Not only are they monitoring your content, but they're actively using your network in order to disguise your traffic and make it appear legitimate. So, as much as I would like to say that we could geo-block or re-architect things in a way that makes it so that individual organizations, like, manage more of the infrastructure of the internet to help provide a layer of, like, security and authentication there, there's already so much infrastructure out there that relies on these systems of residential proxies that kind of eliminate that possibility unless the entire internet system is re-architected from kind of a fundamental level. WILL: Yes. Yes, that's right. TIM: My favorite thing is when we get a cybersecurity issue, and the first thing I'm told to do is, "Well, just block the IP." It's like, that doesn't do anything, man. You need to, like, not think about security that way [laughter]. "Just block the IP. Like, you guys know they're, like, vended botnets where, like Vivian just said, like, people hijack IoT devices everywhere. So, Mike, your question was: how do we mitigate or try to defend it? First thing is you just have to change the paradigm of your organization, engineers, and everyone to stop looking at AI as an adversary. And you just have to...if you can't beat them, join them kind of thing. You have to have that situation, that acceptance, like, AI is not going anywhere. We can bomb as many data centers as we possibly can. Engineers will just find a way to make AI run on less hardware, you know? We've seen that happen before with DeepSeek and frontier models, like, keep getting distilled so they can run on anyone's iPhone or whatever they want, you know? It's just going to keep getting there. So, we've been tracking a couple of threat categories, and a threat category that originates from AI and how you're supposed to defend against it. So, the first one, AI-generated phishing, right? Phish attacks. That happens so much. And it's not that hard to prevent it; it's just a pain in the ass for everyone who's on the other side. You have to require MFA. You have to require FIDO2. You have to require passkeys. No matter how good your phishing attack is, they're not going to break a FIDO2 passkey, yet. Now, we put a yet there because there is a threat that we're tracking where the lines of quantum cryptography and quantum computing converge with our LLMs, and we're a little worried about that. That's when I start to get a little freaked out because now an LLM has the compute capacity to short-circuit a model or, you know, an RSA cryptographic signature instantly. Password attacks, move to passwordless. Conditional access, MFA, that's how you keep them from doing that. Malware generation that, you know, script kiddies give a Kimi K3 task to write a new malware thing. Well, you have to do application allowlisting. You have to do behavior-based

detection. You know, like John was saying, put, you know, some kind of UBA mechanism on devices to make sure you're not getting attacked. Vulnerability discovery and detection, we've lived in this world for three or four years now, where everything has to shift left. You have to have the SAST, the SCA, the IaC; all of those tests have to come before someone even does a pull request. We want those tests to happen before it even leaves a developer's machine when code is committed. Deepfakes: Apple just pushed this out, this week, Apple Reference Images. Google was the one that actually came up with the technology. You see something that looks questionable, you're like, "Cool, show me your reference image. Show me the signed cryptographic, you know, signature to this image that shows it wasn't modified." That's already out there. There are a lot of things, strategies that are out there in the industry to combat the risks that AI presents. The problem is a lot of businesses are not willing to swallow the heartburn that's going to cause on engineers in the day-to-day. And sometimes people, organizations overreact to some of these threats, and they might make things harder than they need to be. You know, like rotating a password every four minutes when really you should just put MFA on there, you know, that's the kind of mistakes we see a lot of organizations make. WILL: Yeah, but how is this going to affect our scheduled day offs [laughter]? JUSTIN: We aren't even going to talk about, like, how AI is going to replace all of our jobs [laughter]. WILL: You know, I'm very much of the mindset that, you know, if you can't beat them, join them, and that's just fine. But, like, having been on the surviving end of several, let's see, let's say, like, productivity revolutions in software development, you know, of which this is one, this is certainly one. I love it. Productivity revolution, like, great. Great. Super great. I love it. But, man, my ticket queue hasn't gotten any easier. Has it [laughter]? And, like, I wouldn't say that, like, I'm not the best around. I'm not the best AI cat around. I've got a couple of friends, you know, who are real hotshots on it, and that's...God bless them. But I'm by no means a Luddite, and yeah. Yeah, I'm going to be working hard right up until the final bell on a Friday. Count on it [laughter]. MIKE: So, we talked about...So, joining them. So, you got to have your observation layer, your watchers, right, watching this. And nobody can afford to have somebody on every watchtower, right? So, you have to automate this. You have to have the AI engaged to be doing something with the AI. You bring up something else. I think it was mentioned earlier that your average big org takes a month and a half to patch a serious vulnerability. Your vulnerability policy that you had from last year isn't going to work anymore. So, there's my second suggestion. You got to start caring about those vulnerabilities and patching them now. You thought that you could wait till next month when you had a little more time; I don't think that's true anymore. TIM: So, corollary, I mean, downstream effect from that is that you have to have a way to test that your patches are, you know, good. And if you have a QA department that's going to take you a couple of weeks to make it through that, that's not going to be good. So, you got to automate all your tests and, you know, get your pipeline up and running to where you can do that very rapidly. Now, a lot of people already have this, but a lot of people don't. WILL: I feel like the automation of QA, I've always had a very soft spot in my heart for QA just because I've seen so many difficult things to diagnose, and troubleshoot, and validate, and, like, a lot of every organization has a lot of secret sauce. And I feel like...I do feel like the pendulum, it's about time for the pendulum to start swinging back the other way with QA because those guys have been...It's been a bloodbath in QA for the past several years, and, I think, we cut too deep industry-wide. I'll say it, 10 toes down [laughter]. JUSTIN: So, I almost think that, I mean, just as an aside, kind of a response to that, I think, QA engineers who were click QA engineers who didn't really do that much beyond clicking and such, I think that position is gone. But I think QA, true QA engineers who understand end-to-end and who are automating things and who are dedicated to their craft, I think, that position is more needed than ever. WILL: Exactly. Exactly. And when I say the QA engineer, like, what I'm talking about is somebody who knows how everything works end to end, flow to flow, like, all of the whole process of this thing going back and forth, and all the things that have exploded, and all the things that are so easy to fall through the cracks between engineering teams, where it's just like, "Oh, well, I work on the checkout cart, so that's my problem." "And I work on the front end; that's my problem." "And I work on the lease, you know, information underwriting data collection part, so that's my problem." I've known so many people in QA who were the final UX authority from, like, this entire flow from soup to nuts, and I don't see a replacement for that, and I don't see that value becoming less. But you're right; I wholly agree that if you are just a button monkey, you're probably going to need to tool up, or you're going to need to retrain one way or the other [chuckles]. TIM: We've looked at this idea of if I, the cybersecurity team, find a vulnerability in your software, I'm going to verify it, confirm it, give you as much information as I can to an engineer or a leader so that they know, "Yep, this is a vulnerability. This is where it needs to be fixed." But then on our side, we're also going to write kind of like what you consider a unit test in software engineering, but it's a cybersecurity test. It's a scripted PoC designed to test that your

vulnerability does exist, and then we would automate that. There are a lot of tools out there, like, Nuclei and other frameworks where I can write a security exploit that's to the PoC level only. And what we want to do is, every time you do a new version out to prod, that server is going to go through and do a regression test to make sure that your new feature didn't disable some kind of security vulnerability that we patched last year. And that idea of QA from a cybersecurity perspective is kind of a little bit new in that we're designing an automated hacker to make sure the things that you patched don't come back up, or that you actually did fix the vulnerability. It's kind of a fun idea, and we're still trying to brainstorm through it, but yeah. WILL: And so, I'm curious, like, what are you guys seeing in the wild, to the degree that you can, you know, disclose? Obviously, that's not, like...Obviously, that's a privilege to a degree, but, like, just on a broad, qualitative, you know, sense, what are you seeing, like, dealing with stuff out in the wild? TIM: The OPSEC side of that question...[laughs] I will say a lot of the attacks that I am seeing are not terribly sophisticated. What's happening is the volume at which they're coming at you as an organization is overwhelming. So, I'm seeing a lot of vended botnets because, like Vivian said, you know, Roku's...Oh man, there was even a free TV box (I'll have to pull it up. It was on a Darknet Diary podcast episode.) that gave people free TV, but it was a harvester in everyone's homes for a botnet to use, and man, they had such a huge botnet. So, we're seeing a lot of volume where threat actors are just doing everything they can to get past the credential layer of your application. We're not seeing a lot of RCEs. We're not seeing a lot of memory escapes. We're not seeing a lot of the higher computer science PhD-level of exploits. We're seeing a lot of the threat actors relying on engineers not to do the basics. Like, if you have a basic username and password, you will be attacked. If you don't have MFA or something like that enabled on your login profile, you will be attacked. So, they're just looking for the low-hanging fruit that you have not done the minimum to step up to. WILL: I'll say...I think I can be vague enough about this. Something that I have seen, and I know for a fact this is AI-enabled, I have heard about a large telecom company who was getting attempts made on them from a reverse engineer of their mobile app, whereby they are trying to exploit...We're talking about the decompilation, right? And they're reverse engineering their mobile app to go out and run exploits on non-published APIs. So, there's an API that's still in the oven. It's released behind a feature flag, right? Because, like, you can't roll a mobile app the same way you can roll a web app, right? You've got to, like, put it out there, and then when the back end is ready or, like, your release channel [inaudible 50:16], you turn it on. Well, I have heard of people decompiling the mobile app, right, and just, like, filtering through and just seeing what they can see, taking this thing apart and, like, finding non-published APIs and trying to exploit those. Now, the people that I'm talking about are good boys and girls, and they, you know, treat the mobile application as wholly untrustworthy [laughs]. TIM: Zero trust. WILL: Code that was written by us kind of, and is running on somebody else's machine and doing God knows what, you know, because a mobile app is wholly untrustworthy. But I believe that is a direct line to an AI-enabled attack, just because applications are under active development, and they get released very quickly. And the chore of decompiling the apps is relatively burdensome, you know, to just go and filter through it and see if there's anything you can see. So, it wasn't a problem, but it was just like, "Oh, wow, that's a funny story that this person told me [laughter]." MIKE: Well, again, I think we can be...We have seen requests coming in from an emulator as the agent [laughs] the mobile app is running on. Wait, what? You should not be running this in an emulator. That is not right. When we did a new release of the mobile app, you know, there's clearly something untoward going on there. There is no reason one of our customers, a week after we released the mobile app, should be running on an emulator [laughs] and not on an actual device. You know -- TIM: That's just poor...I'd call that poor cyber-attacking. MIKE: I agree [laughter]. TIM: They control those headers. They should not have... MIKE: Exactly. TIM: They shouldn't have had it be, like, an emulator header. MIKE: That's what I thought [laughter]. WILL: Like, just do me the courtesy [laughter], you know what I mean? Like, at least put it through Opus for crying out loud. That's some haiku level shit [laughter]. MIKE: So, one more thing I'd like to propose is that we should be proactively attacking our own things. TIM: Yes. MIKE: You know, we've talked about the defenses, the watchers, but it doesn't matter how good you think your wall is; if you haven't tried to knock it down, you really don't know. So, you know, Netflix famously built their Chaos Monkey. TIM: Monkey, yeah. MIKE: So, they would just turn off important parts of their infrastructure at random. And that enabled them to have the infrastructure that it could handle things that, you know, handle when things went down by accident. They did it on purpose. If we are not taking a similar approach of actively attacking our code, our infrastructure, everything about our stuff, I don't think that we are going to be prepared for the coming onslaught. Thoughts? TIM: That is a requirement today. Just getting really nice tools that the bad guys are going to have and using those to attack your own stuff, that's a big deal. I love this podcast called Darknet Diaries. The first of this

month, they released an episode called The Courthouse, where it was all...these were physical pen testers. Their mission was to break into a courthouse and to see how far they can get into the courthouse, what records they can get to. And this is a small town, and the small-town sheriff, they did end up tripping an alarm, so, like, okay, you know, they walk out, and they have a signed article from the mayor's office, whatever, saying, "Hey, we were authorized to do this penetration test." And I guess the sheriff just had a bone to pick with anyone who would try to get into one of his federal buildings, and he sent them to jail no matter what. He just had this prejudice against, you know, "We would never do a penetration test. That's...You don't do that. Why would you do that?" And they did, like, these interviews of people in the community, like, "That makes no sense. Why would you do that?" And the mentality is absent, even in organizations where we think, you know, a routine scan or an audit is sufficient. We actually need to be way more involved and way more aggressive in how we test our systems. We need to get really ugly and dirty because the other guys have no qualms of crossing those boundaries [laughter]. JUSTIN: Yeah. And just leave your pride at the door because you are going to be, you know, hit every which way, and they're going to find some really ugly stuff; at least, hopefully, you find it yourself. But... TIM: Yeah. I had a threat actor hit me personally. I've got some public infrastructure for my own stuff that I've got out there. This guy did some open-source intelligence on me, and he figured out my dad's name. And so, he started sending me phishing emails where the...when you send an email, you can display the name of the sender email, and it can be different than what the actual originator email is. And so, it showed my dad's name as the sender, and then it made a reference to some vacation pictures. First of all, we don't go on vacations as a family. That was, like, my first yellow flag. I was like, what the...[laughter] We are homebody librarian people. We don't share photos. We don't care [laughter]. Like, it just threw me so quickly [laughter]. I was like, "What is this?" But it was coming from my dad, and I was like, "Wait." So, then I pull the source; I look at it. I was like, "This is one of the best phishing attempts I've ever seen." It was a curated phishing attempt to me based off the information around my OSINT out there, and I was like, "Damn, dude, you earned it. That's really good." Like [laughter], that is incredibly good. And I was like, well, I guarantee a business would fall for this. Like, you see someone from a real email that you trust, from a name that you trust with a personal reference, that could be just as simple as, "Hey, I've got a meeting at this time. Here's the Webex link." Cool. It pops up right there, boom. You own them. JUSTIN: You bring up a really good point that a lot of us

have parents who are older, and, in fact, I need to go chat with my dad later on today because his phone got owned, and I got to walk him through a factory reset, but that is a whole different subject for another day. But, like, older folks who can get deepfaked, you know, and, all of a sudden, all of their retirement money is gone, and that is just a scary thing. And they're being deepfaked from a photo of their kid who, you know, a professional photo, or something like that, and, you know, they have some details from a vacation or something like that, and it's gone. WILL: Man, all I'll say is, like, we as an industry run on the backs of lonely, overworked single men a very long way from home, and, I think, we should be more...I think we should be aware of more classical espionage honeypot scams because, "Oh, sweetie, I'm your long-distance Canadian girlfriend, and man, I've been trying to get this program working on my computer. Honey, you're so smart. Can you fix it for me [laughter]?" TIM: Yeah. Romance scams and, what's it, pig fishing or... JUSTIN: Spear phishing? WILL: Spear phishing. Yeah. Yeah. I don't know. I've been working with these guys for a long time. Some of them are a little pent up [laughter] making bad decisions. MIKE: So, final point there is that humans are often the weak link. It doesn't matter how good your external security is, just takes one of said people opening one email in desperation. WILL: I think the industry has been going through a contraction and running a skeleton crew extremely hot, and it's been this way for several years. This has been a rough business cycle for engineering, and we've got all of this sort of...We could see the wave gathering in front of us. And, you know, the cynic in me, having survived many of these business cycles in my lifetime, is some people are going to get swamped before we start making changes because, like, they are systemic. There are systemic aspects to the industry that this is going to...The tide is coming out, and there have been a lot of people who have been cutting a lot of corners for a long time. And me saying it and everybody nodding their heads and saying like, "Yep, you're right," is not going to make any kind of difference. Like, people are going to have to get got. And, I don't know, I guess I did my job, and I said, "Hey, guys, I don't know about this [laughter]." MIKE: We could probably go on for hours here. I think that we've covered some key tactics you can take, but if we don't do it now, it's going to be too late. WILL: Yeah, that's right. We have watchmen. We have watchmen. But he sleeps [laughter]. So, no watchmen while he sleep. What could we do? Only have watchmen, must sleep [laughter]. MIKE: I know that we are being proactive, and, you know, we've recently staffed up, and I'm optimistic that we're going to be making some right choices. I also am fully confident that in the reality of software, there are going to be vulnerabilities somewhere. There will be a library somewhere that [chuckles], even if our software is perfect, there will be a library somewhere. There'll be a vulnerability found, and as we said before, they're minutes once that's exploited. So, we have to, as ourselves at Upbound, and as an industry, change our posture here. So, here's the voice of warning [laughter]. Act now; act today. The barbarians are at the gates. Sometimes they're already in the gates, and there's no time left to wait. TIM: And on that happy note...[laughter] MIKE: Exactly. And I'm leaving it bleak here. This is a call to action. I'll leave [chuckles] it there. Till next time on the Acima Development Podcast.