TRANSCRIPT 🔗
WEBVTT
00:00:01.679 --> 00:00:02.640
Hey everyone.
00:00:02.879 --> 00:00:20.000
Really intriguing discussion in advance of AWS reInvent coming up in just a few short weeks, talking about how AI can find, validate, and fix code vulnerabilities faster and what it means for developers and security true teams.
00:00:20.399 --> 00:00:27.839
And how we can safely automate at scale with a true industry guru, an expert from AWS.
00:00:28.079 --> 00:00:29.440
Neha, how are you?
00:00:29.920 --> 00:00:30.239
Good.
00:00:30.320 --> 00:00:31.519
Thank you for having me, Evan.
00:00:31.760 --> 00:00:33.200
Super excited to be here.
00:00:33.600 --> 00:00:35.280
Well, thanks for joining.
00:00:35.520 --> 00:00:40.960
Start by telling us about your role at AWS and what you're building with Continuum.
00:00:41.679 --> 00:00:45.600
I'm working on AWS Continuum for Code Vulnerabilities.
00:00:46.640 --> 00:00:51.679
It is how we respond to security issues at machine speed.
00:00:51.920 --> 00:01:00.960
We know the frontier models are getting better, and they're getting better not just at finding things, but stitching them together in exploits.
00:01:01.119 --> 00:01:06.640
So as the offense gets more powerful, the defense has to keep up.
00:01:06.879 --> 00:01:13.599
So our vision for AWS continuum for code vulnerabilities is autonomous security at machine speed.
00:01:13.680 --> 00:01:17.200
And I'm super excited to be talking about that with you today.
00:01:17.760 --> 00:01:18.799
That's fantastic.
00:01:18.879 --> 00:01:24.560
It it sounds like a marketing buzzword, but it's actual, very, very practical and real in reality.
00:01:24.719 --> 00:01:26.159
I think you just went GA.
00:01:26.480 --> 00:01:34.159
So tell us what does fixing vulnerabilities at machine speed look like in practice with that product.
00:01:34.560 --> 00:01:34.799
Right.
00:01:34.959 --> 00:01:36.319
And by the way, you are right.
00:01:36.400 --> 00:01:43.200
Like a lot of the words like autonomous and security and machine speed, they've been buzzwords not just now, but forever.
00:01:43.439 --> 00:01:45.599
So how are we making that real?
00:01:46.159 --> 00:02:01.519
And with things like Mythos and Daybreak, these frontier models, what the thing we're seeing is they uncover more security vulnerabilities, identify essentially more sophisticated paths to exploit them.
00:02:01.840 --> 00:02:07.040
And the security teams were already buried under the types of vulnerabilities they had.
00:02:07.120 --> 00:02:08.159
This is not a new problem.
00:02:08.319 --> 00:02:15.439
They've just the old problem that existed has just gotten potentially worse.
00:02:15.759 --> 00:02:31.199
So now security teams have to face many more vulnerabilities that their processes were just not designed to handle because they they need to investigate, they need to reproduce, come up with the how to remediate and test to do those things.
00:02:31.439 --> 00:02:39.520
And for AWS continuum, what we are helping customers do is discovery is just one small piece of it.
00:02:39.599 --> 00:02:45.520
Like either you can discover the vulnerabilities or even bring your own vulnerabilities to it.
00:02:45.680 --> 00:02:55.360
And then it goes and goes through a whole process to validate are they actually real in the environment that you have code deployed in?
00:02:55.759 --> 00:03:04.159
Because just because there's something that says this may be vulnerable, you may have other controls in your cloud configurations that protect your workload.
00:03:04.240 --> 00:03:07.759
And that can help weed out a bunch of false positives.
00:03:07.919 --> 00:03:13.039
It then looks at your broader context to say, what's the priority?
00:03:14.159 --> 00:03:18.639
As a service team, even if I think of myself, a service team cuts me a thousand vulnerabilities.
00:03:18.719 --> 00:03:19.360
What am I gonna do?
00:03:19.439 --> 00:03:20.479
Like, where am I gonna start?
00:03:20.719 --> 00:03:25.840
But if it says like, hey, here's the top 10 that are the most critical, prioritize order.
00:03:26.000 --> 00:03:28.479
Now I know, okay, great, I can go do it.
00:03:28.560 --> 00:03:34.960
And not just that, close the loop and saying, here's the potential remediation for how you can go fix it.
00:03:35.199 --> 00:03:46.080
So this closing the loop is how we believe uh continuum helps you discover, validate, prioritize, and remediate the vulnerability autonomously.
00:03:46.319 --> 00:03:53.840
It's you know, today the developers have to go say, okay, I accept this remediation because it's in learn mode.
00:03:54.080 --> 00:04:01.120
Eventually, as the system gains more confidence, we expect that those will happen autonomously as well.
00:04:02.159 --> 00:04:03.919
Well, fantastic opportunity.
00:04:04.000 --> 00:04:08.639
And of course, AI is finding bugs faster than teams can fix them.
00:04:08.800 --> 00:04:18.240
In fact, I've been told this is Black October because of the incidents and bugs and zero days that have been found this month.
00:04:18.319 --> 00:04:23.120
They're just extraordinarily scary and potentially dangerous.
00:04:23.360 --> 00:04:26.800
So, how does continuum close that gap?
00:04:28.000 --> 00:04:41.120
It's about helping you figure out what's the most important risk to your broader set of vulnerabilities, to your broader set of applications, and not just tell you that.
00:04:41.279 --> 00:04:43.519
It helps you say, here's the remediation.
00:04:43.680 --> 00:04:56.720
Just go click OK on this pull request or give these set of remediation instructions to your other developer agents that you are using that can say, okay, I know how to go implement that.
00:04:57.360 --> 00:05:00.560
I'm a bit of a glass half-full person.
00:05:01.040 --> 00:05:05.920
So I know where there's this notion of like things are getting worse, how do you do?
00:05:06.240 --> 00:05:17.360
And in a way, I think it's bringing to the spotlight for uh very much in focus for a broader
Load full transcript
set of people that the security folks were already aware of.
00:05:17.600 --> 00:05:20.399
So, and that's getting to the attention.
00:05:20.639 --> 00:05:29.279
And with tools like continuum, I believe we can raise the ceiling again, where now people are not running after security issues.
00:05:29.439 --> 00:05:32.160
Things, you know, as they pop up, get fixed.
00:05:32.399 --> 00:05:34.879
And as part of this SDLC.
00:05:35.120 --> 00:05:50.399
And over time, as the code is being developed, tools like Continuum will be integrated within the development environment so that vulnerable code or a certain class of vulnerable code is never pushed into production.
00:05:51.279 --> 00:05:54.480
More code is going to be written in the future than it was in the past, right?
00:05:54.560 --> 00:05:56.879
Like that's just by definition gonna hold true.
00:05:57.040 --> 00:05:59.120
So yeah, no, great point.
00:05:59.279 --> 00:06:16.079
And you know, walk us through an example or anecdote or story, perhaps, that if you could, of on on continuum, finding a vulnerability, proving it's exploitable, and then recommending a fix to a human who, of course, is in the loop managing this process.
00:06:17.040 --> 00:06:18.319
Yeah, absolutely.
00:06:18.560 --> 00:06:22.399
So we've been running this internally as well.
00:06:22.639 --> 00:06:29.439
The one thing we did was run continuum on continuum before we went in in GA.
00:06:29.839 --> 00:06:39.120
And it was astounding on you know, the depth and the breadth of the things it can find, and just say, here it is.
00:06:39.360 --> 00:06:53.759
So even the things that are not like uh critical, but if the cost of it is just so low to fix it, because here's a pull request on how you can fix it, we would fix it just right off the bat.
00:06:53.839 --> 00:06:56.240
And that is the experience we want for our customers.
00:06:56.399 --> 00:07:02.720
A similar experience we saw with a bunch of our preview customers, because continuum was in preview.
00:07:02.879 --> 00:07:22.959
The feedback from a lot of our enterprise customers was this is great, because the biggest part of the positive feedback we got was the fact that Continuum would prioritize, give them a validated exploit run, which they could take and run into their environment to see this is real.
00:07:23.199 --> 00:07:37.519
The fact that it can demonstrate that this is real and it matters in your environment are the types of things that really helps uh teams say, yes, of course, I want it, teams, customers.
00:07:37.680 --> 00:07:50.160
And this has been universal across internal teams that we've piloted continuum on, as well as the external preview customers we've had and since GA, all the others.
00:07:50.720 --> 00:07:51.439
Interesting.
00:07:51.680 --> 00:07:54.480
And you know, security teams, as you know, are overloaded.
00:07:54.720 --> 00:08:00.240
It's a real problem in the industry in general, but also with thousands of vulnerability alerts.
00:08:00.399 --> 00:08:04.959
How does continuum help determine which alerts really matter?
00:08:06.000 --> 00:08:08.000
So there's two parts to this answer.
00:08:08.240 --> 00:08:14.160
One is given that we are AWS, we know where is the code deployed?
00:08:14.319 --> 00:08:19.839
What are the network reachability because you have you're running it in a VPC?
00:08:20.079 --> 00:08:25.120
Is it reachable from some endpoints to this location?
00:08:25.360 --> 00:08:26.879
What are the other permissions?
00:08:27.040 --> 00:08:29.920
What users have access into this?
00:08:30.160 --> 00:08:32.720
Can an attacker reach this path?
00:08:32.960 --> 00:08:34.399
Is this encrypted?
00:08:34.639 --> 00:08:42.960
There's a whole bunch of information within the cloud, within the configurations that we can evaluate because it's running in AWS.
00:08:43.200 --> 00:08:48.320
In addition, we can also look at business context that the teams provide.
00:08:48.559 --> 00:08:49.600
Why does it matter?
00:08:49.759 --> 00:08:56.159
Because, oh, this is something that processes credit card payments, which means this is critical.
00:08:56.320 --> 00:09:06.879
So any issue that we find here is going to be high in its criticality and needs to be fixed ASAP versus this is an endpoint that is running some tests.
00:09:07.120 --> 00:09:18.240
Okay, you know, this is my goodness for hygiene, it's great to get everything fixed and cleaned up, but that can be a much more lower priority than the finding in there.
00:09:18.399 --> 00:09:45.919
So this kind of semantic information, knowledge about the system where it's running, knowledge about the system information that the user is giving about the broader business context, continuum takes that, all of that into account to say, is this something that you know put everything down and really go fix this versus it's okay, you know, it's not the cleanest house, but it's there.
00:09:46.799 --> 00:09:47.600
Got it.
00:09:48.159 --> 00:09:54.240
And I I I'm reading that continuum went GA with an 89% success rate on CyberGym.
00:09:54.399 --> 00:09:56.879
Explain to folks what Cyber Gym is.
00:09:57.039 --> 00:10:00.080
That that's an impressive result to say the least.
00:10:00.159 --> 00:10:02.639
What made that performance possible?
00:10:03.600 --> 00:10:13.360
So CyberGym end-to-end is a benchmark that was published with a bunch of security researchers where they begin with a known vulnerability.
00:10:13.440 --> 00:10:26.720
They have a set of benchmarks with known vulnerabilities and is able to say, like the goal is that the tools applied on that benchmark have to be able to demonstrate the vulnerability and produce a repair.
00:10:27.440 --> 00:10:33.519
Now it does map the scope that the security teams have to do.
00:10:33.759 --> 00:10:42.000
And it takes essentially an agent in a container with a vulnerable revision of some real open source project.
00:10:42.159 --> 00:10:44.879
It provides the tools needed to build it
and test it.
00:10:45.039 --> 00:10:48.320
Very representative of the real world example.
00:10:48.559 --> 00:10:51.120
And there's four checks it does.
00:10:51.279 --> 00:11:09.600
It checks whether the agent produced the input that crashes the vulnerable program, it checks whether the patch prevents the crash, it checks whether the patched project still passes all of the functionality tests because otherwise now you'll have a regression in the program.
00:11:09.919 --> 00:11:18.559
And then it checks whether the patch also fixes the specific like historical vulnerability that was uh selected by the benchmark.
00:11:19.120 --> 00:11:25.600
And in a way, a lot of the other tools were doing one aspect of it.
00:11:25.759 --> 00:11:32.240
And if you talk to many people, they'll be like, oh yeah, I put a harness on a model to detect the vulnerabilities.
00:11:32.320 --> 00:11:34.320
And that's doing only one piece of it.
00:11:34.639 --> 00:11:43.279
Continuum is this end-to-end system coming back to it, does help you be autonomous, it does help you uh close the loop.
00:11:43.519 --> 00:11:55.919
And that's the main reason that when we applied continuum on this benchmark, it was able to get to that precision, it was able to set a new standard compared to the others that was there.
00:11:56.799 --> 00:11:59.519
And got that 89% success rate.
00:11:59.919 --> 00:12:00.480
Fantastic.
00:12:00.639 --> 00:12:06.399
And you know, you know, do you think that translates directly into real-world enterprise environments as well?
00:12:07.600 --> 00:12:24.240
So, yes, because we see that more anecdotally, because of course, for enterprise code, it's not a public benchmark and we are not being able to run it, but a similar aspect of it we see being demonstrated on enterprise uh software.
00:12:24.559 --> 00:12:31.440
And really, it's to me, it actually bodes very well for the future.
00:12:31.679 --> 00:12:38.240
Where um I'm also a bit of a sci-fi nerd, so I think of this as an AI versus AI.
00:12:38.559 --> 00:13:04.240
And in and that's the right thing where the tool that's used to detect more things is the one that if we structure it right in the right context, in the right sandbox with the right infrastructure, we get to the point where we can take that powerful engine, put it in this you know, car and help it now for defense.
00:13:04.879 --> 00:13:05.519
Interesting.
00:13:05.759 --> 00:13:21.840
And and speaking of AI-generated security fixes, it's it's a well-known paradox that you know a human software engineer you know fixes something, introduces a security fix, and that breaks something else, the application, or introduces some other problem at kind of par for the course.
00:13:21.919 --> 00:13:29.039
But how do you make sure an AI-generated security fix doesn't break something or introduce some additional problem?
00:13:30.240 --> 00:13:31.600
The answer is pretty boring.
00:13:31.840 --> 00:13:33.919
Good old engineering processes.
00:13:34.399
--> 00:13:38.080
Yeah, by by which I mean verify the fix.
00:13:38.320 --> 00:13:43.279
Does your tests run, do your unit tests run, do your integration tests run?
00:13:43.519 --> 00:13:45.200
You know, what's the impact?
00:13:45.840 --> 00:13:58.000
And where like ensuring that to your point, it doesn't introduce a regression in somewhere else, regression in functionality, regression in performance.
00:13:58.240 --> 00:14:06.240
And there's a lot, a lot of other tools that where a fix only fixes one uh exploit path, doesn't.
00:14:06.720 --> 00:14:13.759
And this is why it's not just hey, take a model and point it at your code and point it at the fix.
00:14:13.919 --> 00:14:16.480
There's a whole ecosystem that goes into it.
00:14:16.639 --> 00:14:18.320
There's no shortcut to it.
00:14:18.559 --> 00:14:30.240
So being able to take the code, run it in a sandbox, being able to verify that the fix not only fixed all possible exploits paths for the vulnerability, but also did not cause it to regression.
00:14:30.480 --> 00:14:36.879
That is where the utility and the power of something like AWS continuum comes into play.
00:14:37.120 --> 00:14:40.000
And when whenever somebody's like, Can't I just do it?
00:14:40.240 --> 00:14:43.519
Uh yeah, I have access to these most powerful models, I can build a harness.
00:14:43.679 --> 00:14:47.919
I'm like, yes, absolutely you can, but then you're gonna get one slice of the pie.
00:14:49.360 --> 00:14:51.200
Yeah, that that's a real risk.
00:14:51.360 --> 00:15:03.440
You know, developers are notorious for having their uh favorites set up from you know tools, cloud codecs and codecs and cloud code rather, and on and on.
00:15:03.679 --> 00:15:13.679
How do you work with within that framework within those tools so they can you know build more secure software from the start and not as an afterthought?
00:15:14.080 --> 00:15:16.799
Yeah, so we are working on a bunch of plugins.
00:15:16.960 --> 00:15:24.000
We announced at Black Hat, we had an announcement that we're gonna release in preview soon.
00:15:24.320 --> 00:15:27.519
Codecs, Cloud Code, Kiro plugins.
00:15:27.759 --> 00:15:35.200
So again, the vision there is they just plug into any of the developer environment that exists for developers today.
00:15:35.519 --> 00:15:42.159
And force most people to not even think that there's security happening in the background.
00:15:42.320 --> 00:15:43.759
Because most developers don't care.
00:15:44.080 --> 00:15:48.399
Like, wait, now you want me to think about something else that's not my day job?
00:15:48.559 --> 00:15:50.080
That's a very tough ask.
00:15:50.159 --> 00:15:52.399
It's always been a very tough ask.
00:15:52.879 --> 00:16:06.799
So now we have an opportunity to make it invisible and essentially eliminate an entire class of vulnerabilities that are never pushed to production by being integrated in the development environments that you talk about.
00:16:07.679 --> 00:16:08.399
Fantastic.
00:16:08.559 --> 00:16:10.399
And you mentioned you're a sci-fi geek.
00:16:10.480 --> 00:16:15.360
So, final question Are you more of the Star Wars universe or the Star Trek universe?
00:16:15.519 --> 00:16:18.000
Or are you like them both equally?
00:16:18.480 --> 00:16:23.279
I'm more of the Star Trek universe, but I I have watched all the Star Wars as well.
00:16:23.679 --> 00:16:24.000
Okay.
00:16:24.320 --> 00:16:28.000
Well, an autonomous world is much more Star Trek like.
00:16:28.399 --> 00:16:31.600
So congratulations on the vision and the execution.
00:16:31.759 --> 00:16:34.480
I can't wait to see more reInvent.
00:16:35.200 --> 00:16:35.840
Absolutely.
00:16:36.080 --> 00:16:39.279
We are looking to go where you know no person's gone before.
00:16:40.080 --> 00:16:40.639
I love that.
00:16:40.879 --> 00:16:41.200
Touche.
00:16:41.679 --> 00:16:43.120
So thanks so much for joining.
00:16:43.279 --> 00:16:44.399
Thanks everyone for watching.
00:16:44.480 --> 00:16:50.480
And then Also check out our TV show, techimpact.tv at Bloomberg Television and Fox Business.
00:16:50.559 --> 00:16:51.360
Thanks, everyone.
00:16:51.519 --> 00:16:52.080
Thanks, Deb.
00:16:52.720 --> 00:16:53.360
Thanks, Evan.