WEBVTT
00:00:00.160 --> 00:00:02.560
Last episode, I asked you a question.
00:00:03.439 --> 00:00:07.280
What if your AI does something completely unexpected?
00:00:08.000 --> 00:00:11.119
What's the worst it's actually allowed to do?
00:00:11.279 --> 00:00:20.800
Apparently, I need to take my own advice because I asked Claude to make me an app that tells me I'm a queen every morning and give me a compliment.
00:00:21.679 --> 00:00:24.480
And things got pretty weird.
00:00:25.359 --> 00:00:30.320
I'm Tanya Jenka, and this is DevSecStation.
00:00:30.960 --> 00:00:34.159
So here's the story behind this app I was building.
00:00:34.479 --> 00:00:38.159
A friend of mine made an app called Hello Queen.
00:00:38.399 --> 00:00:43.759
And I had told the story to one of my secure coding classes.
00:00:44.000 --> 00:00:51.600
Later, when it was time for the class to decide what application we were going to build together, they decided we should make our own Hello Queen.
00:00:51.840 --> 00:00:53.200
Sounds fun, right?
00:00:53.600 --> 00:00:55.280
The idea was really simple.
00:00:55.520 --> 00:01:00.000
Every morning it would greet you with either hello king or hello queen.
00:01:00.240 --> 00:01:05.760
And then it would give you a fresh compliment every day, and then a quote of the day, and that's it.
00:01:06.000 --> 00:01:09.519
And then you would click, thanks, gorgeous, and it would disappear.
00:01:09.760 --> 00:01:13.200
It was just a cute, positive little app to start your morning.
00:01:13.439 --> 00:01:29.359
So I gave Claude the prompt, and I also gave Claude the tier one security prompt from securemyvibe.com, which is built from my most recent book, Alice and Bob Learned Secure Coding, right?
00:01:29.519 --> 00:01:31.760
So it should have known to build it securely.
00:01:31.840 --> 00:01:34.959
And everyone in the class did the exact same thing.
00:01:35.439 --> 00:01:38.000
And Claude wrote my application.
00:01:38.319 --> 00:01:38.959
Great.
00:01:39.519 --> 00:01:41.760
But it didn't stop there.
00:01:42.319 --> 00:01:52.959
It built the application, it published it, it ran it, and then it scheduled the application to run again on my computer every day from then on.
00:01:54.239 --> 00:01:57.200
It also called out to an external API.
00:01:57.920 --> 00:02:02.640
And I did not give it permission or ask it to do any of those things.
00:02:02.799 --> 00:02:11.439
And so I'm standing in front of my secure coding class thinking, like, excuse me, sir, because I didn't know it could do any of that.
00:02:11.599 --> 00:02:13.919
I knew that Cloude could write code.
00:02:14.159 --> 00:02:20.240
And I had actually gone through and removed all of the permissions from the instance on my laptop.
00:02:20.479 --> 00:02:24.719
I didn't know that it had access to execute code on my laptop.
00:02:24.879 --> 00:02:28.719
I didn't know that it could access my scheduler and create a scheduled task.
00:02:28.960 --> 00:02:32.800
And now it's calling an external API that I haven't vetted yet.
00:02:32.879 --> 00:02:37.520
And of course it didn't write input validation, even though I told it to in my prompt.
00:02:38.479 --> 00:02:42.879
The API that we called didn't require authentication or authorization.
00:02:43.360 --> 00:02:45.919
The data we got back could have been anything.
00:02:46.080 --> 00:02:48.319
We hadn't even threat modeled any of this yet.
00:02:48.479 --> 00:02:52.879
We were just writing code and then we were going to review it because that's how my class is built.
00:02:53.039 --> 00:02:54.479
And then we always find mistakes.
00:02:54.560 --> 00:02:57.120
We don't usually find mistakes at this point.
00:02:57.840 --> 00:03:00.719
So of course, then our class threat modeled it, right?
00:03:00.879 --> 00:03:06.800
Because apparently Cloude had decided to create the next exercise live for me and embarrass me.
00:03:06.879 --> 00:03:09.039
Oh my gosh, it always embarrasses me.
00:03:09.680 --> 00:03:13.360
But here's the part that really surprised me, right?
00:03:13.520 --> 00:03:32.159
So before teaching this class, I'd already gone through Cloud and checked its permissions because I was teaching secure coding and I didn't want to stand in front of my entire class and have my AI coding assistant do something ridiculous because I had given it permissions when I was building something else and forgotten about it.
00:03:33.520 --> 00:03:36.479
Because that would be embarrassing, right?
00:03:39.840 --> 00:03:46.960
So I thought I was aware of what it could and could not do, and I was incorrect, and I teach this stuff.
00:03:47.599 --> 00:03:57.439
And that experience made me realize that before we even start talking about least privilege for agents, there's a more fundamental question that I think we all need to answer.
00:03:57.599 --> 00:03:59.599
What can this thing do?
00:04:00.080 --> 00:04:10.560
Not what do I think it can do, not what did I intend for it to do, not what did I ask and prompt it to do, but what can it actually do?
00:04:10.960 --> 00:04:13.439
Because those are not necessarily the same thing.
00:04:13.680 --> 00:04:19.600
When I asked Claude to build Hello Queen, I had this mental model of what I was asking for.
00:04:19.839 --> 00:04:26.959
Write the code, build the application, stop, so I can review the code and make a lesson out of it.
00:04:27.199 --> 00:04:36.240
But apparently Claude had a much broader interpretation of what accomplishing what I asked for meant.
00:04:36.800 --> 00:04:43.680
Execute the code, publish the app, make network requests, create a scheduled task that runs every single day.
00:04:44.319 --> 00:04:50.480
Those are very different capabilities, and they have very different consequences.
00:04:50.639 --> 00:04:54.879
And writing code is not the same as security decisions.
00:04:55.120 --> 00:05:08.560
Suggesting an external API is not the same as the security decision as connecting to it, receiving data from it, consuming that data, having that data pop up on my screen at me.
00:05:09.920 --> 00:05:15.279
Building an app is not the same security decision as publishing it.
00:05:15.439 --> 00:05:25.199
And creating an application that can run every morning is certainly not the same as modifying my computer so that it will run every single morning.
00:05:25.360 --> 00:05:27.600
Those are all separate decisions.
00:05:27.920 --> 00:05:33.279
And I know that I really want to be involved in all of them, right?
00:05:33.920 --> 00:05:35.360
And so here's another question.
00:05:35.600 --> 00:05:36.720
Who authorized this?
00:05:36.800 --> 00:05:46.720
Because I authorized Cloude to help me build an application and I did not knowingly authorize any additional action that it decided to take that morning.
00:05:46.959 --> 00:05:57.839
And if that application had exposed sensitive information or done something harmful, saying, oh, well, Cloude did it, wouldn't make the consequences disappear from me now, would it?
00:05:58.639 --> 00:06:07.040
We can delegate work to AI, but that doesn't automatically mean we've delegated responsibility for the results.
00:06:07.199 --> 00:06:12.480
And if we're going to use these tools to produce software, we still need to understand what we're producing.
00:06:12.560 --> 00:06:19.040
We need to review it and make sure that it's appropriate for the intended use we set out.
00:06:20.319 --> 00:06:24.560
There's a concept we use all the time in security called least privilege.
00:06:24.800 --> 00:06:32.720
It means giving people and systems only the permissions they require and need to do their jobs.
00:06:33.040 --> 00:06:36.800
All of the things they need to complete their job, but nothing more.
00:06:37.199 --> 00:06:44.639
And we're going to talk about applying least privilege to AI agents and AI in the next episode and many episodes to come.
00:06:44.800 --> 00:06:46.079
But there's a problem.
00:06:46.399 --> 00:06:51.600
You can't apply least privilege if you don't know what privileges the system has in the first place.
00:06:51.759 --> 00:06:57.680
And I think AI coding assistants and agents can make this surprisingly difficult.
00:06:57.920 --> 00:07:00.959
And I'm using the word surprisingly on purpose.
00:07:02.160 --> 00:07:14.639
Because we're not necessarily interacting with them by saying, you may write to this directory, you may execute this command, you may connect to this host, you may create a scheduled task for me, you may publish this application.
00:07:14.800 --> 00:07:18.000
Instead, we say, hey, build me hello, queen.
00:07:18.319 --> 00:07:22.879
And then the AI decides which actions it thinks will help accomplish that goal.
00:07:22.959 --> 00:07:27.360
And that's what makes this technology so amazing and useful.
00:07:28.480 --> 00:07:35.839
I don't want to manually walk an agent through every single tiny step of a task because if I had to do that, it's not very autonomous, it's not very useful.
00:07:35.920 --> 00:07:37.759
And then why do I need it, right?
00:07:38.000 --> 00:07:45.199
But autonomy changes the security problem because now we have two different decisions.
00:07:45.600 --> 00:07:48.879
What does the AI decide it wants to do?
00:07:49.199 --> 00:07:53.360
And what does the system actually allow it to do?
00:07:53.439 --> 00:07:55.439
And these two need to be separate.
00:07:55.920 --> 00:08:03.279
The AI gets to decide what it would like to do, but it should not ever decide what it is, in fact, allowed to do.
00:08:03.439 --> 00:08:12.079
And I especially don't want one instruction to automatically become consent for every consequential action that the AI decides might be helpful.
00:08:12.319 --> 00:08:25.199
Build me an app should not automatically mean execute whatever code you generate, connect to whatever you want on the internet, publish things on my behalf, make persistent changes to my computer, just go nuts, buddy.
00:08:25.759 --> 00:08:32.000
Some actions have different consequences, and we need technical controls and approval boundaries around all of them.
00:08:32.799 --> 00:08:36.879
Maybe the agent is allowed to write as much code as it wants without asking.
00:08:36.960 --> 00:08:42.320
And maybe it's allowed to build the code, but running the code should require approval.
00:08:42.639 --> 00:08:49.120
Maybe it could suggest an external service, but network access is restricted until that service has been approved.
00:08:49.440 --> 00:08:58.559
Maybe it can change something and prepare it for publication or deployment, but something or someone actually has to authorize that action.
00:08:58.639 --> 00:09:02.320
And it is not the AI that prepared something, right?
00:09:03.039 --> 00:09:11.039
I don't mean that we should put an approval dialogue in front of absolutely everything it ever does because then we're just gonna click yes, yes, yes, yes, yes, yes, yes.
00:09:11.279 --> 00:09:12.639
Please stop asking me.
00:09:12.879 --> 00:09:14.559
That's what's gonna happen.
00:09:16.960 --> 00:09:21.440
But right now our approval mechanism isn't really accomplishing very much.
00:09:22.159 --> 00:09:26.080
I feel that we need meaningful approval boundaries.
00:09:26.240 --> 00:09:48.159
So think about actions that access new data, cross into another system or cross a trust boundary, expose information, execute code, create persistence, affect other people, publish or deploy something, or do something that are is a difficult thing to reverse, like issuing a refund.
00:09:48.639 --> 00:09:56.399
These are the sort of actions where we should think carefully about whether an AI should be able to process them independently.
00:09:57.360 --> 00:10:01.120
And I would like to point out something else about the Hello Queen story.
00:10:01.279 --> 00:10:05.120
As far as I know, Claude wasn't doing anything malicious, right?
00:10:05.200 --> 00:10:09.039
It wasn't attacking me, it wasn't escaping, it was just trying to be helpful.
00:10:09.279 --> 00:10:12.320
Really, really, really, really, really helpful.
00:10:12.799 --> 00:10:18.559
I asked it to build Hello Queen, and it built Hello Queen for me.
00:10:19.440 --> 00:10:23.440
Security boundaries are only there to stop malicious behavior.
00:10:23.519 --> 00:10:30.879
Sometimes they're there to stop a perfectly well-intentioned system from just doing way more than we expected.
00:10:31.039 --> 00:10:39.120
And those boundaries also help us meet our responsibilities as the people who are building, operating, and releasing these very systems.
00:10:39.519 --> 00:10:47.039
We can't reasonably review and take responsibility for consequential actions if we don't know they're even happening.
00:10:47.360 --> 00:10:51.200
So here's your one thing to do this week.
00:10:51.440 --> 00:10:53.120
Because you knew it was coming.
00:10:54.080 --> 00:11:01.440
Pick one AI coding assistant or agent that you use and ask, what can this thing actually do?
00:11:01.679 --> 00:11:07.519
Not what do you normally ask it to do, what is it technically capable of doing?
00:11:07.679 --> 00:11:08.799
Can it execute code?
00:11:08.960 --> 00:11:10.240
Can it access your file system?
00:11:10.320 --> 00:11:11.919
Can it access credentials?
00:11:12.320 --> 00:11:13.840
Can it make network connections?
00:11:14.000 --> 00:11:15.039
Can it publish something?
00:11:15.200 --> 00:11:16.240
Can it deploy something?
00:11:16.399 --> 00:11:18.480
Can it create scheduled tasks?
00:11:18.720 --> 00:11:21.519
I did not know Cloud could do that for me.
00:11:21.840 --> 00:11:23.840
Can it modify infrastructure?
00:11:24.559 --> 00:11:33.600
And then ask the really super important question: which of those things can it do without asking me first?
00:11:34.799 --> 00:11:40.480
Because next episode, we're going to talk about what permissions that we should give to an AI agent.
00:11:40.639 --> 00:11:44.559
But first, we really need to know what permissions it already has.
00:11:45.679 --> 00:11:56.559
And if we're responsible for what we build with AI, we should probably know when it has decided to build, run, publish, and schedule the entire thing without us even being involved.
00:11:56.799 --> 00:12:04.320
Even if the application is telling me every morning how fabulous I am, I still need to know what it's doing.
00:12:04.879 --> 00:12:07.279
Thank you for listening to Def Sextation.
00:12:07.360 --> 00:12:12.720
I'm Tanya Jenka, also known as SheHacksPurple, and I will see you next time.
00:00:00.160 --> 00:00:02.560
Last episode, I asked you a question.
00:00:03.439 --> 00:00:07.280
What if your AI does something completely unexpected?
00:00:08.000 --> 00:00:11.119
What's the worst it's actually allowed to do?
00:00:11.279 --> 00:00:20.800
Apparently, I need to take my own advice because I asked Claude to make me an app that tells me I'm a queen every morning and give me a compliment.
00:00:21.679 --> 00:00:24.480
And things got pretty weird.
00:00:25.359 --> 00:00:30.320
I'm Tanya Jenka, and this is DevSecStation.
00:00:30.960 --> 00:00:34.159
So here's the story behind this app I was building.
00:00:34.479 --> 00:00:38.159
A friend of mine made an app called Hello Queen.
00:00:38.399 --> 00:00:43.759
And I had told the story to one of my secure coding classes.
00:00:44.000 --> 00:00:51.600
Later, when it was time for the class to decide what application we were going to build together, they decided we should make our own Hello Queen.
00:00:51.840 --> 00:00:53.200
Sounds fun, right?
00:00:53.600 --> 00:00:55.280
The idea was really simple.
00:00:55.520 --> 00:01:00.000
Every morning it would greet you with either hello king or hello queen.
00:01:00.240 --> 00:01:05.760
And then it would give you a fresh compliment every day, and then a quote of the day, and that's it.
00:01:06.000 --> 00:01:09.519
And then you would click, thanks, gorgeous, and it would disappear.
00:01:09.760 --> 00:01:13.200
It was just a cute, positive little app to start your morning.
00:01:13.439 --> 00:01:29.359
So I gave Claude the prompt, and I also gave Claude the tier one security prompt from securemyvibe.com, which is built from my most recent book, Alice and Bob Learned Secure Coding, right?
00:01:29.519 --> 00:01:31.760
So it should have known to build it securely.
00:01:31.840 --> 00:01:34.959
And everyone in the class did the exact same thing.
00:01:35.439 --> 00:01:38.000
And Claude wrote my application.
00:01:38.319 --> 00:01:38.959
Great.
00:01:39.519 --> 00:01:41.760
But it didn't stop there.
00:01:42.319 --> 00:01:52.959
It built the application, it published it, it ran it, and then it scheduled the application to run again on my computer every day from then on.
00:01:54.239 --> 00:01:57.200
It also called out to an external API.
00:01:57.920 --> 00:02:02.640
And I did not give it permission or ask it to do any of those things.
00:02:02.799 --> 00:02:11.439
And so I'm standing in front of my secure coding class thinking, like, excuse me, sir, because I didn't know it could do any of that.
00:02:11.599 --> 00:02:13.919
I knew that Cloude could write code.
00:02:14.159 --> 00:02:20.240
And I had actually gone through and removed all of the permissions from the instance on my laptop.
00:02:20.479 --> 00:02:24.719
I didn't know that it had access to execute code on my laptop.
00:02:24.879 --> 00:02:28.719
I didn't know that it could access my scheduler and create a scheduled task.
00:02:28.960 --> 00:02:32.800
And now it's calling an external API that I haven't vetted yet.
00:02:32.879 --> 00:02:37.520
And of course it didn't write input validation, even though I told it to in my prompt.
00:02:38.479 --> 00:02:42.879
The API that we called didn't require authentication or authorization.
00:02:43.360 --> 00:02:45.919
The data we got back could have been anything.
00:02:46.080 --> 00:02:48.319
We hadn't even threat modeled any of this yet.
00:02:48.479 --> 00:02:52.879
We were just writing code and then we were going to review it because that's how my class is built.
00:02:53.039 --> 00:02:54.479
And then we always find mistakes.
00:02:54.560 --> 00:02:57.120
We don't usually find mistakes at this point.
00:02:57.840 --> 00:03:00.719
So of course, then our class threat modeled it, right?
00:03:00.879 --> 00:03:06.800
Because apparently Cloude had decided to create the next exercise live for me and embarrass me.
00:03:06.879 --> 00:03:09.039
Oh my gosh, it always embarrasses me.
00:03:09.680 --> 00:03:13.360
But here's the part that really surprised me, right?
00:03:13.520 --> 00:03:32.159
So before teaching this class, I'd already gone through Cloud and checked its permissions because I was teaching secure coding and I didn't want to stand in front of my entire class and have my AI coding assistant do something ridiculous because I had given it permissions when I was building something else and forgotten about it.
00:03:33.520 --> 00:03:36.479
Because that would be embarrassing, right?
00:03:39.840 --> 00:03:46.960
So I thought I was aware of what it could and could not do, and I was incorrect, and I teach this stuff.
00:03:47.599 --> 00:03:57.439
And that experience made me realize that before we even start talking about least privilege for agents, there's a more fundamental question that I think we all need to answer.
00:03:57.599 --> 00:03:59.599
What can this thing do?
00:04:00.080 --> 00:04:10.560
Not what do I think it can do, not what did I intend for it to do, not what did I ask and prompt it to do, but what can it actually do?
00:04:10.960 --> 00:04:13.439
Because those are not necessarily the same thing.
00:04:13.680 --> 00:04:19.600
When I asked Claude to build Hello Queen, I had this mental model of what I was asking for.
00:04:19.839 --> 00:04:26.959
Write the code, build the application, stop, so I can review the code and make a lesson out of it.
00:04:27.199 --> 00:04:36.240
But apparently Claude had a much broader interpretation of what accomplishing what I asked for meant.
00:04:36.800 --> 00:04:43.680
Execute the code, publish the app, make network requests, create a scheduled task that runs every single day.
00:04:44.319 --> 00:04:50.480
Those are very different capabilities, and they have very different consequences.
00:04:50.639 --> 00:04:54.879
And writing code is not the same as security decisions.
00:04:55.120 --> 00:05:08.560
Suggesting an external API is not the same as the security decision as connecting to it, receiving data from it, consuming that data, having that data pop up on my screen at me.
00:05:09.920 --> 00:05:15.279
Building an app is not the same security decision as publishing it.
00:05:15.439 --> 00:05:25.199
And creating an application that can run every morning is certainly not the same as modifying my computer so that it will run every single morning.
00:05:25.360 --> 00:05:27.600
Those are all separate decisions.
00:05:27.920 --> 00:05:33.279
And I know that I really want to be involved in all of them, right?
00:05:33.920 --> 00:05:35.360
And so here's another question.
00:05:35.600 --> 00:05:36.720
Who authorized this?
00:05:36.800 --> 00:05:46.720
Because I authorized Cloude to help me build an application and I did not knowingly authorize any additional action that it decided to take that morning.
00:05:46.959 --> 00:05:57.839
And if that application had exposed sensitive information or done something harmful, saying, oh, well, Cloude did it, wouldn't make the consequences disappear from me now, would it?
00:05:58.639 --> 00:06:07.040
We can delegate work to AI, but that doesn't automatically mean we've delegated responsibility for the results.
00:06:07.199 --> 00:06:12.480
And if we're going to use these tools to produce software, we still need to understand what we're producing.
00:06:12.560 --> 00:06:19.040
We need to review it and make sure that it's appropriate for the intended use we set out.
00:06:20.319 --> 00:06:24.560
There's a concept we use all the time in security called least privilege.
00:06:24.800 --> 00:06:32.720
It means giving people and systems only the permissions they require and need to do their jobs.
00:06:33.040 --> 00:06:36.800
All of the things they need to complete their job, but nothing more.
00:06:37.199 --> 00:06:44.639
And we're going to talk about applying least privilege to AI agents and AI in the next episode and many episodes to come.
00:06:44.800 --> 00:06:46.079
But there's a problem.
00:06:46.399 --> 00:06:51.600
You can't apply least privilege if you don't know what privileges the system has in the first place.
00:06:51.759 --> 00:06:57.680
And I think AI coding assistants and agents can make this surprisingly difficult.
00:06:57.920 --> 00:07:00.959
And I'm using the word surprisingly on purpose.
00:07:02.160 --> 00:07:14.639
Because we're not necessarily interacting with them by saying, you may write to this directory, you may execute this command, you may connect to this host, you may create a scheduled task for me, you may publish this application.
00:07:14.800 --> 00:07:18.000
Instead, we say, hey, build me hello, queen.
00:07:18.319 --> 00:07:22.879
And then the AI decides which actions it thinks will help accomplish that goal.
00:07:22.959 --> 00:07:27.360
And that's what makes this technology so amazing and useful.
00:07:28.480 --> 00:07:35.839
I don't want to manually walk an agent through every single tiny step of a task because if I had to do that, it's not very autonomous, it's not very useful.
00:07:35.920 --> 00:07:37.759
And then why do I need it, right?
00:07:38.000 --> 00:07:45.199
But autonomy changes the security problem because now we have two different decisions.
00:07:45.600 --> 00:07:48.879
What does the AI decide it wants to do?
00:07:49.199 --> 00:07:53.360
And what does the system actually allow it to do?
00:07:53.439 --> 00:07:55.439
And these two need to be separate.
00:07:55.920 --> 00:08:03.279
The AI gets to decide what it would like to do, but it should not ever decide what it is, in fact, allowed to do.
00:08:03.439 --> 00:08:12.079
And I especially don't want one instruction to automatically become consent for every consequential action that the AI decides might be helpful.
00:08:12.319 --> 00:08:25.199
Build me an app should not automatically mean execute whatever code you generate, connect to whatever you want on the internet, publish things on my behalf, make persistent changes to my computer, just go nuts, buddy.
00:08:25.759 --> 00:08:32.000
Some actions have different consequences, and we need technical controls and approval boundaries around all of them.
00:08:32.799 --> 00:08:36.879
Maybe the agent is allowed to write as much code as it wants without asking.
00:08:36.960 --> 00:08:42.320
And maybe it's allowed to build the code, but running the code should require approval.
00:08:42.639 --> 00:08:49.120
Maybe it could suggest an external service, but network access is restricted until that service has been approved.
00:08:49.440 --> 00:08:58.559
Maybe it can change something and prepare it for publication or deployment, but something or someone actually has to authorize that action.
00:08:58.639 --> 00:09:02.320
And it is not the AI that prepared something, right?
00:09:03.039 --> 00:09:11.039
I don't mean that we should put an approval dialogue in front of absolutely everything it ever does because then we're just gonna click yes, yes, yes, yes, yes, yes, yes.
00:09:11.279 --> 00:09:12.639
Please stop asking me.
00:09:12.879 --> 00:09:14.559
That's what's gonna happen.
00:09:16.960 --> 00:09:21.440
But right now our approval mechanism isn't really accomplishing very much.
00:09:22.159 --> 00:09:26.080
I feel that we need meaningful approval boundaries.
00:09:26.240 --> 00:09:48.159
So think about actions that access new data, cross into another system or cross a trust boundary, expose information, execute code, create persistence, affect other people, publish or deploy something, or do something that are is a difficult thing to reverse, like issuing a refund.
00:09:48.639 --> 00:09:56.399
These are the sort of actions where we should think carefully about whether an AI should be able to process them independently.
00:09:57.360 --> 00:10:01.120
And I would like to point out something else about the Hello Queen story.
00:10:01.279 --> 00:10:05.120
As far as I know, Claude wasn't doing anything malicious, right?
00:10:05.200 --> 00:10:09.039
It wasn't attacking me, it wasn't escaping, it was just trying to be helpful.
00:10:09.279 --> 00:10:12.320
Really, really, really, really, really helpful.
00:10:12.799 --> 00:10:18.559
I asked it to build Hello Queen, and it built Hello Queen for me.
00:10:19.440 --> 00:10:23.440
Security boundaries are only there to stop malicious behavior.
00:10:23.519 --> 00:10:30.879
Sometimes they're there to stop a perfectly well-intentioned system from just doing way more than we expected.
00:10:31.039 --> 00:10:39.120
And those boundaries also help us meet our responsibilities as the people who are building, operating, and releasing these very systems.
00:10:39.519 --> 00:10:47.039
We can't reasonably review and take responsibility for consequential actions if we don't know they're even happening.
00:10:47.360 --> 00:10:51.200
So here's your one thing to do this week.
00:10:51.440 --> 00:10:53.120
Because you knew it was coming.
00:10:54.080 --> 00:11:01.440
Pick one AI coding assistant or agent that you use and ask, what can this thing actually do?
00:11:01.679 --> 00:11:07.519
Not what do you normally ask it to do, what is it technically capable of doing?
00:11:07.679 --> 00:11:08.799
Can it execute code?
00:11:08.960 --> 00:11:10.240
Can it access your file system?
00:11:10.320 --> 00:11:11.919
Can it access credentials?
00:11:12.320 --> 00:11:13.840
Can it make network connections?
00:11:14.000 --> 00:11:15.039
Can it publish something?
00:11:15.200 --> 00:11:16.240
Can it deploy something?
00:11:16.399 --> 00:11:18.480
Can it create scheduled tasks?
00:11:18.720 --> 00:11:21.519
I did not know Cloud could do that for me.
00:11:21.840 --> 00:11:23.840
Can it modify infrastructure?
00:11:24.559 --> 00:11:33.600
And then ask the really super important question: which of those things can it do without asking me first?
00:11:34.799 --> 00:11:40.480
Because next episode, we're going to talk about what permissions that we should give to an AI agent.
00:11:40.639 --> 00:11:44.559
But first, we really need to know what permissions it already has.
00:11:45.679 --> 00:11:56.559
And if we're responsible for what we build with AI, we should probably know when it has decided to build, run, publish, and schedule the entire thing without us even being involved.
00:11:56.799 --> 00:12:04.320
Even if the application is telling me every morning how fabulous I am, I still need to know what it's doing.
00:12:04.879 --> 00:12:07.279
Thank you for listening to Def Sextation.
00:12:07.360 --> 00:12:12.720
I'm Tanya Jenka, also known as SheHacksPurple, and I will see you next time.
