Full Transcript for #559: 12 Things You Should (and Shouldn't) Do in AWS

00:04:16.579 --> 00:04:17.320 Morals aside. 00:04:18.230 --> 00:04:18.320 Yeah. 00:04:18.760 --> 00:04:20.140 At least that's my understanding of it. 00:04:20.320 --> 00:04:24.160 Not only is it 100% accurate, it's really fast now when it takes the test. 00:04:24.840 --> 00:04:25.000 Yeah. 00:04:25.280 --> 00:04:25.560 Crazy. 00:04:26.060 --> 00:04:26.200 Okay. 00:04:26.360 --> 00:04:26.420 Yeah. 00:04:26.500 --> 00:04:28.940 I'm really excited about this, this Cloud War Games thing. 00:04:29.020 --> 00:04:34.440 because I've been on the receiving end of waking up to an outage for various reasons. 00:04:34.680 --> 00:04:39.000 And honestly, I think my employments and my applications have gotten better because of it. 00:04:39.140 --> 00:04:41.920 But in the moment, I also aged faster than normal. 00:04:42.840 --> 00:04:43.200 Oh, yeah. 00:04:43.500 --> 00:04:46.080 When you get the C-suite, yeah, it gets intense, 00:04:46.360 --> 00:04:48.760 which is, again, why I tried to create Cloud War Games 00:04:48.800 --> 00:04:52.120 to try and simulate that experience and inoculate people from the stress. 00:04:52.540 --> 00:04:53.420 Yeah, I love it. 00:04:53.540 --> 00:04:54.940 Like I said, it super resonated with me. 00:04:55.220 --> 00:04:58.380 You also authored a course, Zero to Hero, on AWS security. 00:04:58.700 --> 00:05:00.300 Yep, an animated guide security in the cloud. 00:05:00.600 --> 00:05:03.480 So if you like pixel art and you want to learn about AWS security, 00:05:04.200 --> 00:05:08.420 I've got my isometric pixel art there that I demonstrate, you know, 00:05:08.560 --> 00:05:10.820 and animate network maps and show how it all works. 00:05:11.280 --> 00:05:12.940 You've got some really – you also do cartoons. 00:05:13.150 --> 00:05:14.820 These are really fun. 00:05:15.460 --> 00:05:16.460 I like the Black Friday one. 00:05:17.280 --> 00:05:18.680 I like Black Friday stories. 00:05:19.200 --> 00:05:21.320 This Black Friday sale will make or break us. 00:05:21.440 --> 00:05:22.560 Are we really ready on strike? 00:05:22.860 --> 00:05:23.920 Oh, I'm sure we'll be fine. 00:05:24.160 --> 00:05:24.260 Great. 00:05:24.420 --> 00:05:25.560 I thought we'll sell this on then. 00:05:25.840 --> 00:05:26.080 Yep. 00:05:27.000 --> 00:05:29.300 So as you know, that's a huge retail day. 00:05:29.340 --> 00:05:34.560 So if you've got an e-commerce client, maybe it's their first year doing it or the first time they've hit it big. 00:05:35.420 --> 00:05:36.780 And maybe like, oh, we'll be fine. 00:05:36.880 --> 00:05:38.840 It's like, no, you probably should provision up. 00:05:39.080 --> 00:05:48.740 So each year with my clients, we check in and make sure leading up to that that we've got everything provisioned accordingly and plans for if it needs extra, if we've got to double in scale or something like that. 00:05:48.980 --> 00:05:49.160 Yeah. 00:05:49.620 --> 00:05:53.240 I've never had anything go down on Black Friday, but of course, they get quite a bit of traffic. 00:05:53.360 --> 00:05:55.960 And certainly I've almost had it go down. 00:05:56.400 --> 00:05:59.160 And it's often not for the reason you expect. 00:05:59.380 --> 00:05:59.940 You're like, really? 00:06:00.460 --> 00:06:00.980 That's the thing? 00:06:01.220 --> 00:06:02.020 That was the weakest link? 00:06:02.060 --> 00:06:02.880 I had no idea. 00:06:03.380 --> 00:06:03.500 Yeah. 00:06:04.300 --> 00:06:10.440 One time it was, I had extra large JavaScript file that was being served and gzipped by 00:06:10.680 --> 00:06:10.960 Nginx. 00:06:11.240 --> 00:06:12.440 And that almost took down the site. 00:06:12.640 --> 00:06:12.860 Really? 00:06:13.060 --> 00:06:13.180 Okay. 00:06:13.440 --> 00:06:13.860 Oh, well. 00:06:14.220 --> 00:06:15.880 But yeah, that was a fun cartoon. 00:06:15.940 --> 00:06:17.420 So how often do you do these cartoons? 00:06:17.540 --> 00:06:18.100 How many you got? 00:06:18.880 --> 00:06:24.560 I was doing them monthly at one point, but now it's more as it comes to me a little bit 00:06:24.640 --> 00:06:24.780 more. 00:06:25.220 --> 00:06:28.300 So we're, you know, but yeah, there's them. 00:06:28.420 --> 00:06:30.260 I mean, there's 30, 40 of them or so. 00:06:31.020 --> 00:06:33.700 Some of them have gotten on Reddit quite a few views. 00:06:33.800 --> 00:06:36.000 Programmer humor, r slash programmer humor. 00:06:36.320 --> 00:06:36.880 They like it there. 00:06:37.080 --> 00:06:39.180 I am all here for r slash programming humor. 00:06:39.300 --> 00:06:40.320 That place is hilarious. 00:06:40.880 --> 00:06:41.080 Yeah. 00:06:41.420 --> 00:06:41.480 Yeah. 00:06:42.000 --> 00:06:42.180 Amazing. 00:06:42.780 --> 00:06:42.940 Amazing. 00:06:43.480 --> 00:06:43.720 All right. 00:06:43.820 --> 00:06:47.740 Well, let's start by just talking about and like get a high level. 00:06:48.080 --> 00:06:49.540 You know, people want to move to the cloud. 00:06:49.920 --> 00:06:52.160 You know, what are some of the benefits that they see? 00:06:52.400 --> 00:06:55.820 Like what are maybe some surprise issues that they run into? 00:06:56.320 --> 00:07:00.580 I'm sure you've had this conversations with a lot of C-suites and dev leads and so on. 00:07:00.700 --> 00:07:01.660 I'd say the complexity. 00:07:02.340 --> 00:07:07.040 AWS, I think I did a post pretty recently, but AWS has a meeting simulator service now 00:07:07.280 --> 00:07:10.040 where you can train yourself on how to take meetings. 00:07:10.240 --> 00:07:11.520 And I'm like, that's too many services. 00:07:12.560 --> 00:07:14.960 So probably the number one problem is the complexity. 00:07:15.100 --> 00:07:17.580 People are like, oh my gosh, I don't understand this. 00:07:17.640 --> 00:07:19.080 I understand what I'm getting billed for. 00:07:19.480 --> 00:07:23.580 And if you can whittle it down to some core services that meet your needs and your goals, 00:07:24.340 --> 00:07:27.460 then a huge amount of these services aren't even necessary.

00:07:27.640 --> 00:07:30.460 There are services for robotics, which is great if you're in robotics, 00:07:30.530 --> 00:07:33.100 but that's not something you really need to worry about if you're just an e-commerce platform. 00:07:33.600 --> 00:07:36.160 So there's just so much, such a variety there. 00:07:36.400 --> 00:07:39.460 So pick your tools and just get really good with the tools. 00:07:39.900 --> 00:07:42.660 There's some basics that you got to use for everything, which we're going to go over in a minute. 00:07:43.100 --> 00:07:43.320 Yeah. 00:07:43.850 --> 00:07:48.320 One of my most recurring experiences with AWS is I open up the console. 00:07:48.640 --> 00:07:50.540 I'm like, what is all of this? 00:07:50.920 --> 00:07:51.600 What is this? 00:07:51.900 --> 00:07:54.160 So you're suggesting that maybe trim that down a little 00:07:54.400 --> 00:07:55.780 and just put the stuff that you really know 00:07:55.840 --> 00:07:56.500 that you're going to need there? 00:07:56.800 --> 00:07:59.680 Yeah, and we can go over kind of some of my core stacks there. 00:07:59.760 --> 00:08:01.780 It depends if you want to be serverless or not serverless. 00:08:01.880 --> 00:08:02.920 If you're just starting out, 00:08:02.940 --> 00:08:04.360 you don't have a crazy amount of traffic. 00:08:04.560 --> 00:08:05.900 I'd go serverless a lot of times. 00:08:06.080 --> 00:08:07.660 It's real easy to set up, less to manage. 00:08:07.960 --> 00:08:08.560 Okay, yeah. 00:08:08.680 --> 00:08:09.680 What are some of the core ideas? 00:08:09.820 --> 00:08:12.740 I mean, certainly Lambda sounds like that's in there. 00:08:13.000 --> 00:08:15.000 Probably EC2, S3, what are we talking? 00:08:16.219 --> 00:08:17.580 Lambda makes a great web server. 00:08:18.080 --> 00:08:19.900 the API gateway plus Lambda. 00:08:20.740 --> 00:08:22.360 So you have the serverless, 00:08:22.960 --> 00:08:24.840 extremely cheap application layer right there. 00:08:25.070 --> 00:08:25.960 So not a data layer, 00:08:26.470 --> 00:08:29.260 but that's a great start to just connect those two together. 00:08:29.580 --> 00:08:31.140 And if anybody wants, 00:08:32.159 --> 00:08:33.500 these are my offer for free. 00:08:33.530 --> 00:08:34.300 I've got them open source, 00:08:34.479 --> 00:08:35.659 schematical.com slash free. 00:08:35.950 --> 00:08:38.360 I have Terraform scripts that'll set this up. 00:08:38.700 --> 00:08:41.180 So I don't know how chronologically ordered we want to go, 00:08:42.080 --> 00:08:44.860 but I just jump it back a step. 00:08:45.140 --> 00:08:47.140 Terraform or CloudFormation. 00:08:47.400 --> 00:08:48.140 I highly recommend. 00:08:48.210 --> 00:08:51.100 So if you're just getting into it, a lot of times, just like code, right? 00:08:51.130 --> 00:08:53.460 You always want to have your code in some type of version control. 00:08:53.610 --> 00:08:55.820 You don't want to just have it FTP or saved to your disk. 00:08:55.870 --> 00:08:57.760 And you're just like, oh,

you know, it disappeared. 00:08:57.810 --> 00:09:02.740 I don't know when Bob over here, you know, other developer X made this change. 00:09:03.060 --> 00:09:04.480 You want all that in version control. 00:09:04.760 --> 00:09:10.100 And you can do that infrastructure as code known, you know, which is Terraform, Bintofu, 00:09:11.780 --> 00:09:15.800 and CloudFormation is actually AWS's proprietary one that they made themselves. 00:09:16.340 --> 00:09:20.480 And so I'd recommend you get in that habit because it's very difficult to get into that later. 00:09:21.120 --> 00:09:25.820 Once you're you've established your infrastructure, you've all hand spun it up and you're like, how did I do that again? 00:09:26.320 --> 00:09:29.140 What was that? What would what is such and such that left the company? 00:09:29.280 --> 00:09:32.520 And now, you know, so you're basically describing me 10 years ago. 00:09:33.740 --> 00:09:40.100 I was there. I didn't I didn't get on that game till I was a couple of years after they had cloud formation launched. 00:09:40.130 --> 00:09:43.380 At first, I was like, this could be cool. And then I was like, OK, this is essential. 00:09:43.880 --> 00:09:47.940 Yeah. Yeah. I just, I used to be just like, I'm going to SSH in and I'll just get it working. 00:09:48.380 --> 00:09:52.300 And then, you know, the code will be well-structured, but just kind of the server, 00:09:52.520 --> 00:09:56.280 once I get it set up, it'll be fine. And then I realized actually, no, it's not fine. 00:09:57.040 --> 00:10:00.420 You know, you want to make any changes you want to move, you want to move from one setup to 00:10:00.560 --> 00:10:04.120 another. You're like, Oh gosh, I don't even like, this is a research project to figure out what to 00:10:04.220 --> 00:10:09.820 do. Yeah, exactly. But definitely when you're using those, those terraforms, always read your, 00:10:10.200 --> 00:10:11.000 what you're about to apply. 00:10:11.380 --> 00:10:13.900 I've had too many people that just auto apply 00:10:14.360 --> 00:10:15.120 the Terraform changes. 00:10:15.190 --> 00:10:17.960 And what happens is they'll have a slight renaming 00:10:18.100 --> 00:10:18.460 of a database. 00:10:18.570 --> 00:10:19.520 This actually happened recently. 00:10:19.660 --> 00:10:22.380 Somebody had their AI vibe code, this thing, 00:10:22.700 --> 00:10:24.120 and it wanted to rename the database. 00:10:24.730 --> 00:10:27.340 Well then what happens is Terraform deleted the database 00:10:27.700 --> 00:10:29.560 and then spun up another database right there. 00:10:29.590 --> 00:10:30.780 That was their production database. 00:10:31.230 --> 00:10:31.880 That was a rough day. 00:10:32.880 --> 00:10:34.380 That wasn't the project I have. 00:10:34.550 --> 00:10:36.240 I did a write up on it, but it wasn't my project. 00:10:36.390 --> 00:10:38.320 I was always read your replies though, basically. 00:10:38.640 --> 00:10:40.140 Always know what you're about to send. 00:10:41.320 --> 00:10:43.200 Make sure you think it's an update. 00:10:43.390 --> 00:10:45.820 It's got to be an update, not a delete and replace. 00:10:46.300 --> 00:10:49.720 This is something that has certainly been in the zeitgeist lately. 00:10:50.260 --> 00:10:52.280 It's a little bit, I don't hear about it as much. 00:10:52.310 --> 00:10:53.820 I think it's sort of to be dealt with. 00:10:54.110 --> 00:10:57.840 But there's certainly horror stories of the AI deleted the production database 00:10:58.230 --> 00:11:00.600 because kind of like we opened with like, 00:11:00.830 --> 00:11:02.980 instead of trying to just solve the problems, like, is there a better way? 00:11:03.080 --> 00:11:06.780 Like, well, there's a problem inserting to the database 00:11:07.060 --> 00:11:07.860 because the structure is wrong. 00:11:08.120 --> 00:11:11.400 So if we just recreate the database with the right structure, this code will work, right? 00:11:11.500 --> 00:11:15.320 It's like, well, that technically will solve the problem, but that is not acceptable. 00:11:15.779 --> 00:11:16.620 I've heard of it. 00:11:16.780 --> 00:11:18.000 They didn't like the schema. 00:11:18.100 --> 00:11:21.140 So they said, okay, drop the table and I'll create a new one. 00:11:21.540 --> 00:11:27.620 Yeah, I've literally had that happen to me only in my dev machine where it doesn't really matter. 00:11:27.620 --> 00:11:29.180 I'm like, why are there no records? 00:11:29.280 --> 00:11:30.140 Where's all the data? 00:11:30.560 --> 00:11:33.420 Oh, it was better for like, no, we never do that. 00:11:33.580 --> 00:11:37.000 But I'm not giving my AI access directly to my production database. 00:11:37.200 --> 00:11:37.540 No, thanks. 00:11:38.000 --> 00:11:39.560 No, I would not recommend that. 00:11:39.680 --> 00:11:46.160 I mean, as we're seeing more requests for that from like as C-suite people become more functional with these agents, 00:11:46.660 --> 00:11:49.860 they're like, well, I want direct access to the database so I can get the latest in this and that. 00:11:49.920 --> 00:11:53.480 It's like, hold on, let me code you up a tool call, get your read access. 00:11:53.720 --> 00:11:58.920 And if you want to do any right, they'll code up another tool call that'll make it so it's approved by a human. 00:11:59.340 --> 00:12:01.220 But doesn't that, you know, they could say, hey, I suggest this. 00:12:01.220 --> 00:12:03.840 And then, you know, but we've seen a lot of that lately. 00:12:04.600 --> 00:12:08.860 I mean, it's great that we're empowering more people to interact with the data and do more with it. 00:12:08.940 --> 00:12:13.820 You just, we still got to be careful about InfoSec and these things do make mistakes occasionally. 00:12:14.360 --> 00:12:14.960 Yeah, yeah. 00:12:16.040 --> 00:12:19.520 They work in a narrow context and let's make this code run. 00:12:19.600 --> 00:12:20.420 Like, oh, this will fix it. 00:12:20.480 --> 00:12:22.500 Like, we need the whole

company to survive. 00:12:22.780 --> 00:12:31.500 No, I really love your suggestion of maybe creating like an MCP or a tool that's read only and make all the AI access go through that. 00:12:31.600 --> 00:12:38.240 You can observe it and check it out, but you don't have even the capability to destroy it through this mechanism, right? 00:12:38.480 --> 00:12:50.580 Well, there's even an extra layer to that is I just had a post come out about this because I had this request come through is put it pointed at your data lake if you can, not your production database, because those agents can spam like no tomorrow. 00:12:50.700 --> 00:12:58.420 So if they write a very complicated read query that goes through your tool call and they spam it could slow down your production database if it's not, you know, quickly optimized. 00:12:58.880 --> 00:13:02.320 So what we did to make it even more effective is our data lakes. 00:13:02.330 --> 00:13:05.840 A lot of times you don't want to store personally identifiable information and data lakes and all that stuff. 00:13:06.040 --> 00:13:06.900 It's anonymized. 00:13:06.930 --> 00:13:08.500 You can do big data regression on it. 00:13:08.690 --> 00:13:13.820 And so that way, even if it gets data that possibly be personally identifiable, it would be in your production database. 00:13:14.360 --> 00:13:17.020 The data lake theoretically wouldn't have that if you're doing it right. 00:13:17.440 --> 00:13:24.380 And so they couldn't, even if they leaked all your sales data for the last five years, at least you wouldn't have a PII leak, which would cost you even more. 00:13:24.720 --> 00:13:26.680 So that's my recommendation there. 00:13:26.840 --> 00:13:33.320 And I didn't actually, for the outline for today, I didn't talk about data lakes, but I can tell you my favorite data lake stack on AWS, but it's an advanced one. 00:13:33.400 --> 00:13:34.720 I'd save that for number 14, maybe. 00:13:35.020 --> 00:13:36.580 All right, maybe we'll get to that at the end. 00:13:38.200 --> 00:13:38.780 Hey, one second. 00:13:39.680 --> 00:13:43.040 Normally, this would be an ad break from Sentry, but not this time. 00:13:43.400 --> 00:13:46.960 Let's just thank them for supporting the show and get right back to the conversation. 00:13:47.420 --> 00:13:50.840 Also, visit talkpython.fm/sentry after the show. 00:13:51.180 --> 00:13:51.640 Thanks, Sentry. 00:13:52.860 --> 00:14:01.100 Okay, well, let's put a bit of a stake in the ground here and put down the first item as don't hand provision, use Terraform. 00:14:01.300 --> 00:14:11.560 And as I was getting ready for this episode and go through all the stuff we kind of like we'd talk about, I realized that this is acronym packed, an acronym dense and tooled as things. 00:14:11.720 --> 00:14:13.420 So maybe you could help us along the way. 00:14:13.560 --> 00:14:14.620 We've got Terraform. 00:14:15.680 --> 00:14:18.980 And yeah, just maybe a quick summary on this concept. 00:14:19.160 --> 00:14:20.400 Like where we use it and how we use it. 00:14:20.460 --> 00:14:22.520 We've been talking around it, but not as much as an item. 00:14:22.580 --> 00:14:25.360 Safe to assume most of the people listening to this are coders. 00:14:25.960 --> 00:14:26.360 100%. 00:14:26.640 --> 00:14:30.920 Data science coders, yeah, or learners, but, you know, they're still good for them. 00:14:31.080 --> 00:14:31.220 Sure. 00:14:31.240 --> 00:14:37.560 So it's Terraform or IOC infrastructure, IAC, sorry, IAC, infrastructure as code is the category. 00:14:37.800 --> 00:14:42.220 And then under that category, there's Terraform, OpenTofu, and CloudFormation. 00:14:42.420 --> 00:14:48.260 And those are all just ways to define how you want your infrastructure to look basically in code. 00:14:48.480 --> 00:14:52.540 So if I was real simple, I could say I want an EC2 instance, which is... 00:14:52.560 --> 00:14:55.300 just your basic server, virtual server running on Amazon. 00:14:55.900 --> 00:15:00.400 And then I want a database, a MySQL database, this version. 00:15:00.920 --> 00:15:03.100 And I can put those in code in some way. 00:15:03.170 --> 00:15:06.260 And then, like I said, version control it, or I can hit apply. 00:15:06.290 --> 00:15:08.940 So say somebody accidentally deletes my application layer. 00:15:09.040 --> 00:15:10.800 A lot easier to replace than a data layer, of course. 00:15:11.080 --> 00:15:13.840 And I need to spin that back up real quickly. 00:15:13.850 --> 00:15:18.740 You just re-hit Terraform apply, and it re-spins up and provisions the missing area. 00:15:18.830 --> 00:15:21.360 It's really nice, especially for my cloud war games, 00:15:21.500 --> 00:15:25.400 I'm blowing up stuff all the time, intentionally being the, you know, junior that accidentally 00:15:25.820 --> 00:15:26.680 deletes something important. 00:15:27.060 --> 00:15:31.880 A little bit of a code monkey or chaos monkey, rather, running loose kind of thing. 00:15:32.200 --> 00:15:33.360 Yeah, cloud demolition expert. 00:15:33.480 --> 00:15:35.520 I think I had that as my title on LinkedIn for a while. 00:15:35.720 --> 00:15:36.140 I love it. 00:15:36.380 --> 00:15:36.480 Okay. 00:15:36.900 --> 00:15:39.300 Does Terraform do incremental updates? 00:15:39.400 --> 00:15:43.640 Like, let's suppose I've got an EC2 machine and I've got it all set up and it's running 00:15:43.960 --> 00:15:44.820 the app, the database. 00:15:45.240 --> 00:15:50.179 And I'm like, oh, you know, it would be really great if we put these firewall rules specifically 00:15:50.200 --> 00:15:54.680 on the machine or we set up fail to ban or something like that you can define every firewall 00:15:55.000 --> 00:16:00.120 rule which is perfect you wouldn't be able to define as easily what's in the box so to say 00:16:00.340 --> 00:16:05.060 there's you have some control over that um like for example i spin up lambdas and a lot of times 00:16:05.110 --> 00:16:09.340 i have a hello world javascript

i throw in there and it's but i also again these are all for free 00:16:09.340 --> 00:16:13.380 on my website if you want them but um it also spins up the build pipeline and everything like 00:16:13.480 --> 00:16:17.259 that so then you can just hook that up to GitHub but at least it throws in a hello world lambda 00:16:17.280 --> 00:16:21.560 the code. But yeah, you can define all the stuff we're about to talk about today. You can define 00:16:21.660 --> 00:16:31.340 every user role, every access key, every username, the exact way you want your networks to talk. 00:16:31.440 --> 00:16:36.800 So the way your VPC, a virtual private cloud, has its subnet set up, all that stuff can be defined 00:16:37.060 --> 00:16:42.140 in there. It's almost crazy what can be defined. I wouldn't recommend committing your code and 00:16:42.360 --> 00:16:46.539 having it go through there. I'd have that go through a build pipeline separately that you'd 00:16:46.560 --> 00:16:51.260 spin up with terraform or whatever but you'd also you just pull from GitHub sure especially for the 00:16:51.480 --> 00:16:56.200 compiled languages and like go or something where there's no reason to deploy the source with python 00:16:56.400 --> 00:17:01.380 it's almost like not much of a difference although ci is still a thing okay so number one don't hand 00:17:01.560 --> 00:17:05.839 provision use terraform set standards stick to them i love it yeah it's it's getting that habit early 00:17:06.180 --> 00:17:12.299 it's it's tough to get into later number two act don't use access keys use iam roles so again 00:17:12.640 --> 00:17:19.439 acronym please so access access keys so i am is identity access management manager what it's it's 00:17:19.439 --> 00:17:26.240 how you define how your users interact with aws as well as your different running services on aws 00:17:26.680 --> 00:17:31.840 and so a lot of people when they first start out they download their first access keys they ideally 00:17:32.120 --> 00:17:35.780 don't put them in version control where it can be passed around or shared or anything like that 00:17:35.800 --> 00:17:41.120 they do a nice dot env and hope their agent doesn't steal it um but uh you know you get so you get those 00:17:41.080 --> 00:17:44.620 access keys but a lot of times when they push things to production if they're very beginner at 00:17:44.720 --> 00:17:49.280 this they'll put like a.env file in the code or something like that that's got access keys 00:17:49.760 --> 00:17:55.380 and those if those get leaked you know again I've got a story about this that somebody actually 00:17:56.180 --> 00:18:02.920 pushed their access keys for sending email luckily luckily it was just sending and receiving email 00:18:03.340 --> 00:18:07.420 and they pushed it up to a public repo as a junior and they just made a mistake there and it ended up 00:18:07.440 --> 00:18:13.760 on GitHub and some bad guys got it. And I believe we sent 16 million emails for some type of knockoff 00:18:13.880 --> 00:18:18.660 Viagra in about eight minutes before AWS shut down the account and said, what are you doing? It was a 00:18:18.700 --> 00:18:22.000 lot more than we normally said, but that's an example of why you don't want to hard. 00:18:22.280 --> 00:18:30.500 Yeah. I have a thing to share. Have you seen skit? Like, no, it's, it's the pronunciation is perfect 00:18:30.540 --> 00:18:38.460 of what it is, but it's like secrets, S-H-H, get, but get like, oh God. And it used to be a website. 00:18:38.680 --> 00:18:42.500 It's no longer here. Let me see if I can open up this image in a full screen for us to see. 00:18:42.880 --> 00:18:49.680 So it was a website that would scan the real time fire hose of changes on GitHub, pull out all 00:18:50.140 --> 00:18:58.960 cryptographic looking things, AWS keys, you know, email keys. And it just had this stream going and 00:18:58.980 --> 00:19:05.240 it updates, I don't know, like once a second or more. It's absolutely terrifying. And so anytime 00:19:05.820 --> 00:19:09.700 that someone's out there listening and you need to make this point to juniors or people who are new 00:19:09.760 --> 00:19:13.340 or whatever, like do not commit stuff to get it. Like, well, it's a, you know, I can just 00:19:14.020 --> 00:19:18.100 uncommit it. It's not a very popular repo. How fast would people find it? Like this fast. 00:19:18.540 --> 00:19:24.620 Fast enough. Yeah. If it's up there, you should assume it's gone immediately. And so, yeah, 00:19:25.120 --> 00:19:26.420 That's pretty scary. 00:19:26.820 --> 00:19:29.000 You know, I don't know if you tracked this, probably not, 00:19:29.600 --> 00:19:33.800 but at PyCon this year, there was a talk by Tristan McKinnon 00:19:34.340 --> 00:19:36.600 called Zero Trust in 200 Milliseconds. 00:19:37.000 --> 00:19:40.980 Now, I didn't watch the talk, so I'm only going off the abstract. 00:19:41.280 --> 00:19:45.480 But basically, the idea is every time you need to make an API call 00:19:45.580 --> 00:19:46.760 or do a transaction or something, 00:19:46.920 --> 00:19:51.380 you can get a separate 200 millisecond duration key. 00:19:51.960 --> 00:19:53.060 What do you think of that idea? 00:19:53.260 --> 00:19:54.100 I like it. 00:19:54.600 --> 00:19:58.060 200 milliseconds is interesting if that's a set one or a variable one. 00:19:58.330 --> 00:19:58.820 Yeah, I'm not sure. 00:19:58.820 --> 00:20:00.900 It might be a little bit shorter, a little bit longer. 00:20:01.320 --> 00:20:03.720 In case there's a little lag, you don't want to pull three. 00:20:04.060 --> 00:20:06.860 I also immediately wonder how long does it take you to provision that key? 00:20:07.420 --> 00:20:09.640 Hopefully it doesn't take you 200 milliseconds to provision it, 00:20:09.640 --> 00:20:11.200 but if you could provision it in two milliseconds. 00:20:13.060 --> 00:20:17.220 I don't like, what do they call it, chocolate candy infrastructure 00:20:17.400 --> 00:20:21.380 where once

you pass the outside, then it's free once you get inside. 00:20:21.450 --> 00:20:23.700 I forget the exact term there, but you've got to be careful. 00:20:23.880 --> 00:20:25.620 even once you're past the infrastructure, 00:20:26.360 --> 00:20:27.600 the external infrastructure, 00:20:27.810 --> 00:20:28.280 once you're in there, 00:20:28.310 --> 00:20:29.000 you don't want to have it. 00:20:29.020 --> 00:20:30.080 So anybody's got free reign 00:20:30.200 --> 00:20:31.280 to grab whatever they want. 00:20:31.510 --> 00:20:32.720 They might get that. 00:20:32.730 --> 00:20:33.600 So I like it. 00:20:33.880 --> 00:20:34.820 My other question though, 00:20:34.860 --> 00:20:36.200 we always have to balance is, 00:20:36.720 --> 00:20:38.320 what's the latency effects? 00:20:38.720 --> 00:20:40.520 If that costs 200 milliseconds of latency, 00:20:40.700 --> 00:20:41.360 that's pretty expensive 00:20:41.620 --> 00:20:42.460 from a compute standpoint. 00:20:43.100 --> 00:20:43.820 So 00:20:44.240 --> 00:20:45.860 Yeah, I'm not entirely sure. 00:20:45.860 --> 00:20:47.320 But just a little more background about this. 00:20:47.320 --> 00:20:49.160 This is from FedRAMP, 00:20:49.160 --> 00:20:51.500 which is the US government's, 00:20:51.500 --> 00:20:52.860 they use this on a lot of the US government 00:20:53.200 --> 00:20:56.500 internal protection like cloud or whatever, which is maybe a special case. 00:20:56.830 --> 00:21:01.800 That makes a lot of sense. And I wager the government's latency limitations are a lot 00:21:01.830 --> 00:21:04.420 less than someone trying to be competitive in e-commerce or something like that. 00:21:04.600 --> 00:21:07.160 Yes, exactly. You know what? They can wait for security. 00:21:07.520 --> 00:21:11.540 Yeah. It's like the DMV online. Just stand in line all day. They'll get there. There's nowhere 00:21:11.660 --> 00:21:11.960 else to go. 00:21:12.440 --> 00:21:16.780 It's like just get in line with all the other poor browsers and wait back there for your turn and 00:21:16.790 --> 00:21:22.840 we'll get to you eventually. Unless we closed at four. Unless we closed at four. Okay. So 00:21:22.860 --> 00:21:24.700 Basically, don't use access keys. 00:21:24.940 --> 00:21:25.540 Use IAM roles. 00:21:25.940 --> 00:21:26.960 How do we make this happen, though? 00:21:27.120 --> 00:21:32.060 So IAM, you go in there, there's users, there's groups, which makes a lot of sense. 00:21:32.639 --> 00:21:36.900 The roles are things you assign to the various services you boot up. 00:21:37.080 --> 00:21:41.160 So you can have a role for the EC2 instance or a role that the Lambda has. 00:21:41.240 --> 00:21:47.340 So when your Lambda executes, it's acting as if it has the same permissions that this role has. 00:21:47.740 --> 00:21:52.600 So you could say, hey, I want this Lambda to be able to access this one specific S3 bucket. 00:21:52.680 --> 00:21:59.700 you know, and it will basically know these are the permissions it has just because

you've assigned it that role. 00:22:00.080 --> 00:22:05.180 I see. So maybe it can only read from the database and can only write to a certain bucket and that's it. 00:22:05.480 --> 00:22:12.680 It's not network level, it's service level. There's a separate one we'll go over in a minute for network level, how you restrain a network. 00:22:13.180 --> 00:22:18.560 It's more what services can it call on AWS? Can it boot up instances? Can this instance boot up other instances? 00:22:18.620 --> 00:22:20.120 You don't typically want that. 00:22:20.190 --> 00:22:23.140 You know, in this instance, you know, S3 is right. 00:22:23.240 --> 00:22:24.400 Can it invoke Lambdas? 00:22:24.540 --> 00:22:27.220 Can it this, you know, Lambda trigger other Lambdas? 00:22:27.530 --> 00:22:28.920 In some cases that might make sense. 00:22:29.060 --> 00:22:30.780 In some cases that could just be very expensive. 00:22:31.520 --> 00:22:31.580 Yeah. 00:22:31.940 --> 00:22:36.800 But a lot of it, like you could theoretically give a Lambda the ability to boot up or provision 00:22:37.020 --> 00:22:37.200 hardware. 00:22:37.230 --> 00:22:38.540 I don't think you really want to. 00:22:38.680 --> 00:22:41.360 Maybe if you were running a hosting company, but maybe not. 00:22:41.750 --> 00:22:43.500 But it's not a million though. 00:22:43.850 --> 00:22:43.980 Yeah. 00:22:44.140 --> 00:22:44.320 Yes. 00:22:44.540 --> 00:22:48.180 It's a service to service, not network level permissions thing. 00:22:48.330 --> 00:22:48.460 Okay. 00:22:48.600 --> 00:22:54.220 I see. So that's right. We're talking, IAM defines what it can do within AWS as infrastructure, 00:22:54.250 --> 00:22:58.280 the service type of thing, or code as infrastructure, not what can it do within 00:22:58.410 --> 00:23:02.260 your database row level type of things. That's, that's correct. Yeah. The database engines, 00:23:02.920 --> 00:23:07.860 you know, wouldn't specify that Dynamo is a little bit of a Dynamo's serverless has got invokes. 00:23:08.040 --> 00:23:13.760 It's not like something like a Redis or, you know, or MySQL where it expects traffic on a 00:23:13.870 --> 00:23:17.760 certain port, you know? So there's some of their proprietary stuff. It actually does have a little 00:23:17.740 --> 00:23:19.640 extra power there. 00:23:19.700 --> 00:23:23.660 But when they're doing open source stuff and they can't really change the 00:23:23.780 --> 00:23:27.400 innards of it, then it just says, can you start a new one or stop this one? 00:23:27.700 --> 00:23:28.380 That's pretty much it. 00:23:28.480 --> 00:23:28.600 Okay. 00:23:29.020 --> 00:23:33.880 And following on from there, don't use generic IAM permissions. 00:23:34.240 --> 00:23:34.900 Use granular ones. 00:23:35.040 --> 00:23:36.340 Like, just let it do everything. 00:23:36.540 --> 00:23:37.000 That'll be easier. 00:23:37.180 --> 00:23:37.440 It works. 00:23:37.720 --> 00:23:38.620 That could trade the permission errors. 00:23:39.080 --> 00:23:39.520 Oh,

yeah. 00:23:39.660 --> 00:23:41.420 So let's go back to that email one. 00:23:41.600 --> 00:23:44.960 What if they said, let it do anything on any service right there? 00:23:44.980 --> 00:23:47.240 They could have deleted everything in a second. 00:23:47.660 --> 00:23:51.480 people are like, okay, that makes sense. But what, what I see is for convenience sake, you're, you're 00:23:51.820 --> 00:23:54.140 banging your head against the wall. And all of a sudden you're like, you know what, we're just 00:23:54.180 --> 00:23:57.960 going to let them access all the S3 buckets. And next thing, you know, a couple, you know, 00:23:58.400 --> 00:24:03.080 months later or whatever, they, they've now got your internal reporting and they pull that out, 00:24:03.140 --> 00:24:09.280 you know, it's like, nope. So don't ever do the asterisk is your enemy. Basically S3 colon asterisk 00:24:09.340 --> 00:24:13.540 gives you all S3 permissions. You know, you could delete that bucket right there and then it's down. 00:24:13.960 --> 00:24:16.940 So avoid using the asterisk whenever possible. 00:24:17.140 --> 00:24:18.000 Same thing with resources. 00:24:18.550 --> 00:24:20.640 So it's not just permissions, but it's also the resources. 00:24:20.770 --> 00:24:22.680 Like I said, you've got 20 S3 buckets. 00:24:22.920 --> 00:24:24.900 Specify only the ones you want to write to. 00:24:24.910 --> 00:24:27.020 You can actually specify down to the path if you want to. 00:24:27.160 --> 00:24:28.860 Sounds like if you use the asterisk, 00:24:28.870 --> 00:24:31.420 it sounds like a good way to end up on the front page of a computer 00:24:31.600 --> 00:24:32.800 or somewhere you don't want to end up on. 00:24:33.000 --> 00:24:33.240 Yeah. 00:24:33.560 --> 00:24:36.400 So, I mean, if you're experimenting for 10 seconds 00:24:36.560 --> 00:24:37.180 and you're going to delete it, 00:24:37.260 --> 00:24:38.440 I mean, I wouldn't commit it into version control. 00:24:38.620 --> 00:24:39.120 Let's put it that way. 00:24:39.120 --> 00:24:40.000 You want your version control, 00:24:40.180 --> 00:24:42.180 if you're like accidentally poke a hole for 10 minutes, 00:24:42.300 --> 00:24:43.560 to be like, okay, how is this not working? 00:24:43.860 --> 00:24:48.500 You want your version control, Terraform, IIC to wipe that out the second you're done with it. 00:24:48.670 --> 00:24:49.000 Makes sense. 00:24:49.020 --> 00:24:49.980 Probably not do that in production. 00:24:50.260 --> 00:24:50.420 Yeah. 00:24:50.740 --> 00:24:51.900 It's worth repeating a couple of times. 00:24:52.250 --> 00:24:56.500 You know, it sounds to me like you're pretty savvy with the agentic stuff and so on, 00:24:56.790 --> 00:24:58.900 given your tool calling and things you were talking about. 00:24:59.260 --> 00:25:04.140 I feel like five years ago, granular permissions and things like that were really tricky because 00:25:04.190 --> 00:25:05.980 you're just like, ah, what do I need? 00:25:06.060 --> 00:25:08.460 Like so many boxes for allow or disallow. 00:25:08.780 --> 00:25:14.320 Now with agents, I feel like you can just say, look, I need to do this with this service to that service. 00:25:14.680 --> 00:25:15.980 Probably be pretty accurate. 00:25:16.040 --> 00:25:17.660 What are your thoughts on using that for help? 00:25:17.900 --> 00:25:19.320 Yeah, it depends. 00:25:19.620 --> 00:25:25.740 So just imagine you're not an extremely technical person and you tell the agent, listen, I just need this to talk to that. 00:25:26.040 --> 00:25:28.140 They can achieve that by putting that asterisk in there. 00:25:28.520 --> 00:25:33.560 So, you know, so that's an interesting one where I've seen them to accomplish their goal. 00:25:33.660 --> 00:25:36.020 They don't exactly go about it the way you'd want them to. 00:25:36.420 --> 00:25:39.360 But I've also seen it counteracted by people using multiple agents. 00:25:39.790 --> 00:25:44.980 And AWS does offer its own DevOps agents and FinOps agents that are going to put me out of a job and all that. 00:25:46.320 --> 00:25:53.800 But I would say, I mean, this is kind of, so I think I mentioned beyond Cloud War Games, I'm writing a game that's going to be on Steam. 00:25:54.160 --> 00:25:59.280 And the goal is to make sure that people with the least amount of technical skills understand all the bad things that can happen. 00:25:59.670 --> 00:26:04.120 You know, because you might just, right now we're seeing with the agents is that these people are getting incredible half hours. 00:26:04.420 --> 00:26:06.380 But for 30 years, I've been programming. 00:26:07.540 --> 00:26:09.360 And some people, it looks like Wizardry. 00:26:09.390 --> 00:26:10.480 Now they've got that power. 00:26:10.670 --> 00:26:12.480 They don't know the vulnerabilities. 00:26:13.120 --> 00:26:14.500 It can go really poorly for you. 00:26:14.650 --> 00:26:17.160 And so the agents, I mean, it can be great. 00:26:17.560 --> 00:26:21.260 But man, you got to know what a cross-site scripting attack is. 00:26:21.500 --> 00:26:24.380 Because if it's not validating that, what is a MySQL injection attack? 00:26:25.720 --> 00:26:26.740 And it's rocky. 00:26:26.980 --> 00:26:29.360 So it's great that they can speed you up and make you more efficient. 00:26:29.560 --> 00:26:32.100 But make sure you're double-checking it to work, is what I guess I've got to say. 00:26:32.400 --> 00:26:34.400 Yeah, and as I'm hearing you talk, I'm just thinking, 00:26:34.900 --> 00:26:38.280 this is just one more reason that being an actual expert 00:26:38.720 --> 00:26:40.980 and using these tools is so much more powerful 00:26:41.200 --> 00:26:43.300 than just trying to vibe your way through it. 00:26:43.440 --> 00:26:47.300 So the using the star thing, that is the easiest way to solve it, 00:26:47.580 --> 00:26:48.760 and that might be what the agent does. 00:26:48.880 --> 00:26:51.360 But you could say, and your primary goal 00:26:51.560 --> 00:26:55.280 is

to be absolutely least privileged about this thing. 00:26:55.480 --> 00:26:57.520 And so you're going to win this session 00:26:58.020 --> 00:27:01.660 by getting the least privilege that you possibly can now go. 00:27:01.840 --> 00:27:03.300 Like that would make all the difference, I imagine. 00:27:03.540 --> 00:27:05.360 I would think having competing agents, 00:27:05.740 --> 00:27:08.260 one that's the coder and you give it that instruction, 00:27:08.370 --> 00:27:10.600 but then have a second one do a peer review on it 00:27:10.980 --> 00:27:12.460 might be of some use to be like, 00:27:12.600 --> 00:27:14.620 hey, is there any possible way to break this? 00:27:14.920 --> 00:27:17.260 Just one agent alone, I don't, I'm here. 00:27:17.540 --> 00:27:21.120 From my experience, it'll just say, 00:27:21.120 --> 00:27:22.460 yeah, sure, I did it sometimes. 00:27:22.550 --> 00:27:23.480 Yeah, yeah, yeah, 100%. 00:27:23.650 --> 00:27:25.920 Having competing more skeptical agents seems to help, 00:27:26.040 --> 00:27:26.940 but if you get the part, you know. 00:27:27.140 --> 00:27:29.780 Right, ask for an adversarial review or something like that. 00:27:29.920 --> 00:27:30.100 Yeah. 00:27:30.400 --> 00:27:33.180 All right. Don't put back-end resources on public subnets. 00:27:33.540 --> 00:27:35.880 Do you need your database on the internet just open? 00:27:36.400 --> 00:27:36.800 Probably not. 00:27:37.030 --> 00:27:40.160 Exactly. This is one of the things where it's not convenient because people are like, 00:27:40.260 --> 00:27:43.080 well, I just want to be able to hit my database for my local machine. 00:27:43.240 --> 00:27:45.780 Well, you still can. You've got a bastion, if anybody's familiar with that. 00:27:45.810 --> 00:27:46.800 I can explain that in a second. 00:27:46.980 --> 00:27:50.160 But let me zoom out to, there's a thing called a virtual private cloud. 00:27:50.340 --> 00:27:52.060 It's basically inside your AWS account. 00:27:52.460 --> 00:27:54.280 It's a way to kind of siphon off resources. 00:27:54.500 --> 00:27:57.720 Some people, personally, I like having a prod AWS account, 00:27:58.020 --> 00:28:00.200 the staging AWS account and the test. 00:28:00.230 --> 00:28:01.340 And that way you're all separate. 00:28:01.960 --> 00:28:04.760 I'll be honest, for my own stuff, I actually have separate VPCs instead. 00:28:04.850 --> 00:28:07.460 So inside of that, you've got your groups of subnets. 00:28:07.720 --> 00:28:10.340 So this is tough to do without diagramming it right now. 00:28:10.480 --> 00:28:13.260 But you know that there's US West, right? 00:28:13.420 --> 00:28:14.180 There's US West 1. 00:28:14.660 --> 00:28:16.960 And that is a giant bunker in the ground. 00:28:17.170 --> 00:28:20.220 Inside of that, there's about six availability zones. 00:28:20.700 --> 00:28:25.080 And so each one of those availability zones is its own bunker inside that facility that 00:28:25.420 --> 00:28:27.180 has completely independent power supply and all that. 00:28:27.320 --> 00:28:31.240 So if one of those gets new, you still got three others or four or five others in that region. 00:28:31.500 --> 00:28:33.600 Honestly, they keep adding them. So I don't know what they're at exactly. 00:28:33.740 --> 00:28:37.160 But and so inside of those, you want to have subnets. 00:28:37.500 --> 00:28:39.980 Now, a lot of times people put everything in big subnet. 00:28:40.120 --> 00:28:45.480 That's not good. You know, if you just have one subnet on one availability zone in one region, that's not reliability. 00:28:45.960 --> 00:28:51.820 That's not redundancy. And those availability zones independently go down actually fairly frequently. 00:28:51.900 --> 00:28:59.240 But you don't see it if you use multiple availability zones or if you got the budget for it, multi-region, which has actually come down and cost quite a bit. 00:28:59.620 --> 00:29:01.760 So I guess I'm going to split this one into two parts. 00:29:01.810 --> 00:29:06.840 One, use multiple availability zones whenever possible because it'll give you a lot more uptime. 00:29:07.060 --> 00:29:12.660 Two, I always have it so there's a public subnet, which is things that should be accessible from the Internet. 00:29:13.060 --> 00:29:14.800 OK, that'd be the load balancer. 00:29:15.010 --> 00:29:21.200 A lot of times the load balancer takes in traffic and then sends it to various instances running in the various AZs. 00:29:21.500 --> 00:29:33.500 So that's always out there. But then privately in the private subnet, you want to have your databases and you're like, OK, well, I should probably should have done the next one before this because you can firewall these things. 00:29:33.870 --> 00:29:36.800 But man, it's so easy to poke a hole in those firewalls when it's convenient. 00:29:36.930 --> 00:29:44.380 So a lot of times I'll come back and do an audit and something that's like poke the hole here so you can get traffic from anywhere on port 22. 00:29:44.460 --> 00:29:46.220 You shouldn't have that going on. 00:29:46.220 --> 00:29:54.380 You want to only have, you know, so just adding that, having a private subnet specifically for that, for the databases, for anything secure, 00:29:55.140 --> 00:30:00.940 it just adds an extra level of inconvenience that even a valid user who might have been compromised accidentally, 00:30:01.140 --> 00:30:07.000 like you installed a wrong game on your computer, you know, and it grabs your SSH keys and pokes through the security wall, 00:30:07.230 --> 00:30:11.660 and it pokes through the firewall because someone put it on 00, basically allowed the whole world in on that port. 00:30:11.900 --> 00:30:16.680 Well, now they still can't get to it because there's not actually a network path that would go from point A to point B. 00:30:16.860 --> 00:30:19.380 So it's just another way of adding more security to it. 00:30:19.560 --> 00:30:29.360 And listening on the private cloud addresses versus listening on

just 000, you know, like certain things like that are definitely worth considering, right? 00:30:29.600 --> 00:30:30.800 Yeah, well, yeah, that makes sense. 00:30:30.820 --> 00:30:32.480 I should have put this one before the next one. 00:30:32.540 --> 00:30:35.080 Let's jump to the next one and I'll kind of circle back on this one. 00:30:35.300 --> 00:30:36.340 I got a little backwards here. 00:30:36.500 --> 00:30:36.940 No worries. 00:30:37.480 --> 00:30:38.220 So this one? 00:30:38.520 --> 00:30:39.840 Well, we went one too far. 00:30:40.140 --> 00:30:40.500 One too far. 00:30:40.840 --> 00:30:41.160 There we go. 00:30:41.280 --> 00:30:43.200 Don't use one security group for everything. 00:30:43.640 --> 00:30:43.880 You're smart. 00:30:44.400 --> 00:30:44.520 Yes. 00:30:44.860 --> 00:30:48.080 So security group is how we limit network traffic. 00:30:48.180 --> 00:30:48.980 Think of that as firewall. 00:30:49.480 --> 00:30:49.640 Okay. 00:30:49.940 --> 00:30:55.520 This EC2 instance, this Lambda, whatever it is, can access this other thing, this database 00:30:55.840 --> 00:30:58.320 on 0.336, whatever, you know. 00:30:58.320 --> 00:31:00.840 So that says that only those two things can talk. 00:31:01.040 --> 00:31:02.420 It would be a security group rule. 00:31:02.420 --> 00:31:05.780 So what you have is you assign a security group to your Lambda and a security group to the 00:31:06.100 --> 00:31:06.300 database. 00:31:07.080 --> 00:31:11.240 But a lot of times people will just say anything and all these security groups can take traffic 00:31:11.260 --> 00:31:15.480 other one because you just have security group a and anything in security group a can take traffic 00:31:15.560 --> 00:31:21.780 over any port in there so now you've got potentially you know http traffic uh getting that could access 00:31:21.960 --> 00:31:25.780 something else or hit you know hit your database on a port that's not supposed to be hit you know 00:31:25.780 --> 00:31:31.400 so you've got to be very particular with those security group rules um because you don't want it 00:31:31.700 --> 00:31:36.360 so someone that say say you've got a box an ec2 instance and a nefarious party somehow get something 00:31:36.360 --> 00:31:41.480 on there even something the npm remote exploit stuff that's been going around a lot i don't know 00:31:41.480 --> 00:31:45.280 if you've heard about that but oh yeah let's just say you install it in your application layer now 00:31:45.420 --> 00:31:49.260 that application layer do you want it just to be i mean it's all gonna have to talk to your database 00:31:49.800 --> 00:31:55.100 they'll get access to that that sucks but let's just say that that happens in test at least this 00:31:55.200 --> 00:31:59.160 one you know if you had the security group set up right and the subnets and everything it couldn't 00:31:59.240 --> 00:32:03.260 access production you know so that would save you a little bit of headaches but you right don't want 00:32:03.260

--> 00:32:05.920 so they can crawl every IP within your subnet. 00:32:06.100 --> 00:32:06.600 You want that. 00:32:06.840 --> 00:32:09.840 Yeah, I mean, what gets your virtual private cloud network 00:32:10.030 --> 00:32:11.640 if once you're in, you get the whole thing. 00:32:12.220 --> 00:32:15.120 The term that comes to mind for me is lateral movement. 00:32:15.340 --> 00:32:16.400 It's a big thing in security, right? 00:32:16.600 --> 00:32:18.160 Once you're in, you find a way in. 00:32:18.280 --> 00:32:19.500 Well, now where can you go, right? 00:32:19.730 --> 00:32:20.180 Think about that. 00:32:20.380 --> 00:32:20.440 Exactly. 00:32:21.000 --> 00:32:22.920 Lateral movement is a perfect term. 00:32:23.760 --> 00:32:25.260 You also can limit your outbound as well. 00:32:25.350 --> 00:32:27.860 So you can limit where the box can talk out to. 00:32:27.890 --> 00:32:29.040 So if you really want to get secure, 00:32:29.560 --> 00:32:30.800 lock that down as well. 00:32:31.880 --> 00:32:36.600 If only this IP for this one third-party API vendor, that's it. 00:32:36.740 --> 00:32:40.020 They can't go reach out to somewhere overseas that we don't know about. 00:32:40.370 --> 00:32:42.520 So make sure you lock down the outbound if you need to. 00:32:42.830 --> 00:32:45.660 I don't know if I do that as a beginner day one, but if you're a big company, oh yeah. 00:32:47.340 --> 00:32:50.560 This portion of Talk Python is brought to you by Talk Python courses. 00:32:51.160 --> 00:32:52.520 Here's the thing that always bug me. 00:32:52.810 --> 00:33:00.720 You finish one of our courses, that's hours of video, a pile of code you actually wrote, and real skills you didn't have a month before, and then nothing happens. 00:33:01.260 --> 00:33:03.860 No paper, no credential, nothing to show for it. 00:33:04.400 --> 00:33:05.060 So we fixed it. 00:33:05.480 --> 00:33:09.120 Every Talk Python course now generates a completion certificate automatically. 00:33:09.620 --> 00:33:11.580 Go to your account page in your dashboard section, 00:33:12.120 --> 00:33:15.260 scroll down to your completed courses, and click Certificate. 00:33:15.580 --> 00:33:16.360 That's the whole process. 00:33:17.340 --> 00:33:19.960 Two things you can do with these course completion certificates. 00:33:20.400 --> 00:33:24.440 Download the full PDF, which is handy if your employer reimburses training 00:33:24.590 --> 00:33:26.120 or gives you credit for finishing it. 00:33:26.680 --> 00:33:29.980 Or you can make the certificate public and hit Share on LinkedIn, 00:33:30.280 --> 00:33:33.700 which adds it to your LinkedIn profile under licenses and certifications, 00:33:34.340 --> 00:33:35.840 not a poster that scrolls away in a day, 00:33:35.930 --> 00:33:40.240 an actual credential sitting on your profile where your manager and recruiters can see it. 00:33:40.840 --> 00:33:43.000 Plus, if you've been taking our courses for a while, 00:33:43.240 --> 00:33:45.940 you've probably earned several of these without even knowing they existed. 00:33:46.420 --> 00:33:50.280 Just visit training.talkpython.fm/account and collect them. 00:33:51.100 --> 00:33:53.600 Thanks to all of you who have taken a Talk Python course. 00:33:54.200 --> 00:33:55.380 It's a great way to support the podcast. 00:33:56.760 --> 00:34:00.180 What are your thoughts on overlay networks like TailScale and stuff? 00:34:00.240 --> 00:34:07.980 Instead of opening up a port, maybe put your employees on a tail scale network that you can see the into the database server or something, but nothing else. 00:34:09.019 --> 00:34:11.300 So in my terms, a lot of times there's Bastion. 00:34:11.300 --> 00:34:12.700 I think I've got it in here somewhere. 00:34:12.720 --> 00:34:13.120 Yeah, tell us. 00:34:13.350 --> 00:34:13.700 Yeah, yeah. 00:34:13.770 --> 00:34:14.540 Tell us about Bastion. 00:34:14.620 --> 00:34:15.360 We can go ahead and jump to that. 00:34:15.639 --> 00:34:21.720 So Bastion's before you boot up a Bastion in a public IP, a public network that could be accessed publicly. 00:34:21.940 --> 00:34:28.820 Ideally, you'd limit it using security groups and saying, hey, this is the only accept traffic on 22 from this IP address, you know, mine. 00:34:28.860 --> 00:34:31.500 And every time I relocate, I'd have to update the IP address. 00:34:31.960 --> 00:34:36.780 And then you'd SSH into that bastion, and then that would be able to then be your door to the rest of everything. 00:34:37.060 --> 00:34:41.460 So we still do port forwarding if you want to do local development, hit the test database or something like that. 00:34:41.860 --> 00:34:44.720 So I do that quite a bit, but they've created a new tool. 00:34:45.139 --> 00:34:46.780 I'm old. I've been doing this since 2010. 00:34:48.080 --> 00:34:50.060 That's called Secure Session Manager. 00:34:51.040 --> 00:34:59.480 it's the SSM and you can use that to enter in using IAM authentication and that's much more 00:34:59.600 --> 00:35:05.940 secure. So I'd consider something more like that. Also, you don't have the cost of running an EC2 00:35:06.040 --> 00:35:10.980 instance 24 seven act as your bastion. So that's a nice tool to have in there. I guess I didn't put 00:35:11.000 --> 00:35:16.880 that in. Yeah, no worries. Have you heard of Knock Knock? Knock Knock? No. So Knock Knock is this 00:35:17.080 --> 00:35:25.520 interesting thing that the firewalls are blocked like 100% for all the management. Let me see if I 00:35:25.520 --> 00:35:30.700 can get this right. So everything is blocked 100% for the IPs. But if you authenticate to a certain 00:35:30.920 --> 00:35:37.400 point, then the service opens up just for your IP address for the length of that session. You kind 00:35:37.400 --> 00:35:46.860 of like knock on a different way, then it'll let you through the firewall. It's the concept. I 00:35:47.220 --> 00:35:53.540 yeah that would be a good one to have in there i would think um so just firewall allow this

these 00:35:53.720 --> 00:35:58.200 ip addresses or whatever you're like okay every time that port opens temporarily i need it to 00:35:58.380 --> 00:36:02.420 authenticate yeah it's kind of cool i mean that's actually a good feature request for aws to have 00:36:02.640 --> 00:36:08.500 uh durations on their their uh security group rules so you say i only want to do this for an hour 00:36:08.840 --> 00:36:12.760 you know right let's open this up so i can i can debug and diagnose this because i gotta get to it 00:36:12.880 --> 00:36:16.840 apparently but then just shut it back off without me requiring to like fail safe instead of fail 00:36:16.860 --> 00:36:19.380 Yeah, that could be a very useful feature. 00:36:19.670 --> 00:36:20.420 We should send that to you. 00:36:20.670 --> 00:36:21.420 Yeah, let's do it. 00:36:21.780 --> 00:36:22.380 Send it through the podcast. 00:36:22.740 --> 00:36:24.180 I'm sure some folks there are listening. 00:36:25.319 --> 00:36:30.000 Apologies to vegetarians for this next one, but don't nurse your EC2 instances. 00:36:30.340 --> 00:36:31.120 Cattle, not puppies. 00:36:31.600 --> 00:36:33.500 I didn't come up with the phrase, but it sums it up. 00:36:33.500 --> 00:36:34.060 No, I know, I know. 00:36:34.280 --> 00:36:35.160 It sums it up very well. 00:36:35.480 --> 00:36:40.200 So I'm sure you've, in your career, probably had a box somewhere where you were just nursing. 00:36:40.380 --> 00:36:42.680 I mean, very common before they did serverless. 00:36:43.910 --> 00:36:46.700 Or before they did virtual servers, I should say, when you actually had a rack. 00:36:46.780 --> 00:36:51.580 somewhere before aws and then you'd have to call a guy someone kicked over the plug you're in trouble 00:36:51.900 --> 00:36:56.620 but you'd end up i saw this even on aws with ec2 instances specifically i see it a lot where 00:36:56.770 --> 00:37:01.440 someone boots up a couple ec2 instances and they're just this is my baby i've got i've got it 00:37:01.490 --> 00:37:05.780 set up absolutely perfect and if one of those things gets sick or dies and you got to restart 00:37:06.020 --> 00:37:10.300 now you're sitting there like oh how did i how did we set this up before did we you know what 00:37:10.320 --> 00:37:17.840 scripts did we run all that stuff and with both lambda and ecs ecs so ecs is elastic container 00:37:18.120 --> 00:37:23.640 service and i don't know if people are familiar with docker but docker is a phenomenal tool where 00:37:23.780 --> 00:37:28.780 you can basically build the image it's almost like if we're going back in time you're building your 00:37:28.860 --> 00:37:33.780 os and saving it to a disk you just pop that disk and anytime it'll boot up the exact one so if you 00:37:33.840 --> 00:37:37.800 have to kill it 10 times a day you know your servers you if you had to delete your servers 10 00:37:37.820 --> 00:37:41.460 times a day you could spin them up exactly the same way they were in that perfect state when

they 00:37:41.490 --> 00:37:47.860 when they first started taking traffic in a split second and it's it's really nice and same lambdas 00:37:48.020 --> 00:37:52.820 that with some simplicity to it um it's it's a lot like docker but it's not actually docker 00:37:52.920 --> 00:37:58.060 under the hood i found out um but it's it's the same thing your lambdas you know will scale 00:37:58.340 --> 00:38:01.880 infinitely especially if you want auto scaling which is again a huge advantage of the cloud 00:38:02.260 --> 00:38:06.220 you'd want to have this type of technology so you can if you're you know it's the middle of the night 00:38:06.140 --> 00:38:08.780 I only want two instances running, two containers, two tasks. 00:38:09.180 --> 00:38:12.320 It's the middle of the day on Thanksgiving, the day before Thanksgiving, 00:38:12.740 --> 00:38:13.680 the day after Thanksgiving, sorry. 00:38:14.060 --> 00:38:15.020 And now we need 100. 00:38:15.460 --> 00:38:16.580 It'll scale just perfectly. 00:38:16.840 --> 00:38:21.560 So be very weary of just SSH-ing in and customizing stuff. 00:38:22.140 --> 00:38:25.280 On EC2, it's just way too easy to do at first. 00:38:25.480 --> 00:38:26.080 It's tempting. 00:38:26.540 --> 00:38:26.900 Yeah, tempting. 00:38:27.700 --> 00:38:32.240 But then once you get to scale, you want to have those images built out 00:38:32.500 --> 00:38:34.980 that'll just spin up very quickly. 00:38:35.620 --> 00:38:36.100 100%. 00:38:36.120 --> 00:38:41.100 long time I resisted using Docker because to me it felt like complexity paired with capacity like 00:38:41.360 --> 00:38:46.540 it's more complex to do Docker and also it's harder for me to observe and analyze it and what I 00:38:46.760 --> 00:38:51.400 really what I realized not too long after that is well all the commands you put in the Docker file 00:38:51.420 --> 00:38:56.640 it's like what you had to write into your server to make it do whatever it does anyway just put run 00:38:56.840 --> 00:39:01.900 instead of not run on the front you know copy versus not copy or whatever and then the observability 00:39:01.920 --> 00:39:06.240 really you can just map some volumes or go into the container like what you can still observe it 00:39:06.350 --> 00:39:11.560 pretty much just as well so i just want to echo your thoughts of like yeah absolutely look into 00:39:11.720 --> 00:39:17.600 docker if you if you want to go fast you know and uptime scalability reliability that's not an issue 00:39:17.980 --> 00:39:22.420 i'm not going to push it you know i i told i was i'm a startup guy i totally get it if you're 00:39:22.740 --> 00:39:28.100 a company that's goes down and loses a hundred thousand dollars an hour and uh you don't have 00:39:28.120 --> 00:39:33.620 the setup commands documented or you know your lead tech gets hit by a bus quote unquote you know 00:39:33.620 --> 00:39:38.360 you want that docker file to be clear as day so you can make modifications spin up whatever so 00:39:38.560 --> 00:39:44.420 100 100 what about kubernetes what do you think on kubernetes versus docker i mean that's another 00:39:44.720 --> 00:39:49.980 level complexity i think another dev i respected i was on a podcast with said if you're if you're 00:39:50.060 --> 00:39:55.380 doing kubernetes on anything but gcp you're you're kind of it's not a good fit it's just you're kind 00:39:55.320 --> 00:40:00.720 of messing around. So they tried, AWS tried, and some people have done real well with this EKS. 00:40:00.980 --> 00:40:07.320 That's the Elastic Kubernetes Service, I believe is what it's called. And that is the ECS's brother 00:40:07.480 --> 00:40:15.800 that runs with Kubernetes. But it just, I've never had it feel natural. I've had clients multiple 00:40:15.980 --> 00:40:21.900 times have me pull them off of EKS to ECS just because ECS is a bit more AWS native and they 00:40:21.840 --> 00:40:26.220 are already on it. So I'm not going to say no, but most of the people I see have the best success 00:40:26.410 --> 00:40:30.380 with Kubernetes are probably using GCP. I have no problem with that. It's not my expertise. 00:40:30.800 --> 00:40:34.220 Yeah, that makes a lot of sense. For Docker, you know, a lot of times, 00:40:34.960 --> 00:40:39.760 it's running yourself, not necessarily in the container service on AWS. You could even do this 00:40:39.880 --> 00:40:43.960 on EC2 for sure. You know, like you can go a long ways with Docker Compose for like people 00:40:44.080 --> 00:40:49.020 startup things like my company. I got 33 Docker containers running on something at this point, 00:40:49.600 --> 00:40:50.620 different distinct ones. 00:40:50.960 --> 00:40:53.900 But anyway, there are other tools like Coolify. 00:40:54.620 --> 00:40:55.200 Have you seen this? 00:40:55.560 --> 00:40:56.800 I've heard of Coolify. 00:40:56.800 --> 00:40:57.660 I haven't dug in much. 00:40:58.040 --> 00:40:58.720 It's pretty interesting. 00:40:58.860 --> 00:41:01.640 So what you can do is you can sign up with Coolify, 00:41:02.040 --> 00:41:03.980 point them at any Linux server. 00:41:04.260 --> 00:41:06.640 It could be EC2, it could be DigitalOcean, Hetzner, whatever. 00:41:07.160 --> 00:41:09.920 And it will install a little agent back 00:41:09.980 --> 00:41:11.420 so they can sort of issue commands. 00:41:11.540 --> 00:41:13.280 And then it will manage all the Docker stuff. 00:41:13.480 --> 00:41:17.900 And it's got like a listing of different services you can pick, 00:41:18.140 --> 00:41:21.500 like thousands of, you know, like I just want to run Hemidol, 00:41:21.980 --> 00:41:24.320 the dashboard for organizing and managing my server applications 00:41:24.720 --> 00:41:25.980 or Hayform or whatever. 00:41:26.130 --> 00:41:28.880 And it'll just, okay, we'll just do whatever Docker things have to happen there. 00:41:29.010 --> 00:41:34.620 And it's a little bit more of I want to take a self-hosted type of thing 00:41:34.770 --> 00:41:36.560 or some kind

of service and just stick it in there. 00:41:36.590 --> 00:41:36.960 I don't know. 00:41:37.200 --> 00:41:38.580 I can completely respect that too. 00:41:38.720 --> 00:41:40.840 There's part of me, trust me, there's a part of me that wants to go off 00:41:40.840 --> 00:41:43.720 and live in a cabin by the woods and have my own servers in there and everything. 00:41:43.980 --> 00:41:47.840 And, you know, self-host, it just, it depends, you know, 00:41:48.000 --> 00:41:50.980 from a business standpoint so a lot of times i work directly you know with the business side of 00:41:51.080 --> 00:41:55.180 people to coordinate with and from a tech side that's really cool from a business side you know 00:41:55.700 --> 00:41:59.920 well what if we someone kicks the power plug you know what if this that and the other like 00:42:00.300 --> 00:42:04.640 how are we going to maintain it you know and there becomes it just depends where you want to invest 00:42:04.800 --> 00:42:09.240 your money and your engineer's time and you could probably i mean you could host you could spin this 00:42:09.300 --> 00:42:14.060 up on a dot on a aws instance and not have to worry about all that stuff have it running in there 00:42:14.080 --> 00:42:18.980 but then the question is how many engineering hours is it going to take to maintain something 00:42:19.120 --> 00:42:20.520 like this versus a managed service? 00:42:20.840 --> 00:42:24.040 Even though it feels like it's managing it for you, it's still a bit of a puppy because 00:42:24.200 --> 00:42:26.320 you still got to do backups and it has a way to do it. 00:42:26.400 --> 00:42:29.880 But if something goes wrong, you still got to, you, it's really down to you to like, 00:42:29.960 --> 00:42:31.640 well, you upgraded and it corrupted the database. 00:42:31.820 --> 00:42:32.760 So now what, you know? 00:42:32.920 --> 00:42:33.060 Yeah. 00:42:33.280 --> 00:42:33.840 Something weird. 00:42:34.120 --> 00:42:35.660 I'm self-hosting N8N right now. 00:42:35.800 --> 00:42:37.600 And it keeps like, oh, you got to do another update. 00:42:37.720 --> 00:42:38.360 You got to do another update. 00:42:38.860 --> 00:42:43.080 How much of my time could I save if I just had N8N on their native platform? 00:42:43.400 --> 00:42:43.600 I know. 00:42:43.760 --> 00:42:46.460 I tried it in for a while and it absolutely was just, 00:42:46.720 --> 00:42:47.960 eventually became a no for me. 00:42:48.360 --> 00:42:49.640 So I've switched over to Hermes, 00:42:49.670 --> 00:42:51.780 which is just ironically on the screen right here, 00:42:51.890 --> 00:42:53.000 which is so good. 00:42:53.680 --> 00:42:55.280 But it's like a sort of self-improvement. 00:42:56.000 --> 00:42:57.220 Oh, I've got Hermes. 00:42:58.400 --> 00:43:00.100 I've got it and used it with mixed results. 00:43:00.300 --> 00:43:03.420 It made me a little angry when I found it was resizing the context. 00:43:04.300 --> 00:43:07.620 It has its own custom AI for changing the context window. 00:43:07.770 --> 00:43:08.900 And so I was losing my context. 00:43:09.090 --> 00:43:10.760 And then this is more of the model's fault. 00:43:10.900 --> 00:43:13.220 The model, I could see the memories it was putting in it. 00:43:13.360 --> 00:43:15.220 You know, it's trying to save in plain text. 00:43:15.230 --> 00:43:18.360 And it was saying the user gets agitated when I, 00:43:19.440 --> 00:43:21.120 it's like a user gets agitated when you hallucinate. 00:43:21.340 --> 00:43:21.740 Yes, but. 00:43:21.900 --> 00:43:22.240 Yeah, exactly. 00:43:22.520 --> 00:43:22.880 When you're wrong. 00:43:22.880 --> 00:43:23.800 It's like a user is agitated. 00:43:24.420 --> 00:43:27.320 Yeah, I have codex and 5.6 soul back in mind. 00:43:27.540 --> 00:43:28.480 So it's pretty smart. 00:43:28.860 --> 00:43:31.040 All right, let's talk logs. 00:43:31.430 --> 00:43:34.120 Because I think when you go away from the single server, 00:43:34.570 --> 00:43:37.820 this starts to be one of the things you worry about is like, 00:43:37.980 --> 00:43:39.160 well, it used to be, 00:43:39.190 --> 00:43:40.740 I could just look at the logs on the machine. 00:43:41.320 --> 00:43:42.900 And now they're all over these different places. 00:43:43.380 --> 00:43:44.220 What do you say about that? 00:43:45.080 --> 00:43:48.440 It takes some getting used to, but it can be your best friend. 00:43:48.820 --> 00:43:50.600 They have incredible tools. 00:43:50.720 --> 00:43:54.740 I was just chasing down a big DDoS attack from one of my biggest customers. 00:43:55.320 --> 00:43:56.900 Almost three times the amount of our normal traffic. 00:43:57.020 --> 00:43:58.380 It was brutal. 00:43:58.700 --> 00:44:03.540 And we're trying to track down like a needle in a haystack there for some other stuff as well. 00:44:03.920 --> 00:44:05.440 And they've got really powerful tools. 00:44:05.640 --> 00:44:08.120 CloudWatch Insights allows you to query those. 00:44:08.280 --> 00:44:09.560 Now, you can spend a lot of money on that. 00:44:09.560 --> 00:44:09.960 Be careful. 00:44:10.040 --> 00:44:13.380 I had a, we taught a guy how to do queries and CloudWatch insights. 00:44:13.830 --> 00:44:15.900 Well, they, it's a, it's a serverless type thing. 00:44:15.910 --> 00:44:19.540 It just charges you for every byte it ingests or the log. 00:44:19.540 --> 00:44:21.960 So if you tell it to go back five years and read through all the logs, 00:44:22.320 --> 00:44:23.840 you're going to have a pretty decent bill there. 00:44:23.840 --> 00:44:24.640 So don't do that. 00:44:24.760 --> 00:44:25.100 Be careful. 00:44:25.500 --> 00:44:27.740 Get your queries figured out in the five minute range and then run them, 00:44:28.060 --> 00:44:31.260 you know, so you're not debugging query over and over again with a long date range. 00:44:32.180 --> 00:44:34.140 But the, and the metrics are super powerful as well. 00:44:34.230 --> 00:44:38.820 You can set logs or you can, they pipe in pretty much everything from

your services. 00:44:39.460 --> 00:44:52.020 I mean, I couldn't even try and label all of them, but the obvious ones are CPU usage, memory usage, latency on requests for your load balancers, volume of requests, et cetera. 00:44:52.140 --> 00:44:55.400 So you can set up these really beautiful dashboards that help me. 00:44:55.920 --> 00:45:06.240 When a client comes to me and says, hey, we've got an extra X seconds of latency spiked on this day, I can go in there and zoom in real deep and just go through layers and go all the way down to the database and be like, 00:45:06.320 --> 00:45:13.060 okay, this database didn't have enough swap or, you know, this Redis eviction shot through the sky, 00:45:13.580 --> 00:45:21.020 all that stuff. So it takes a while to find it, but it pays off so much. And this also goes back 00:45:21.080 --> 00:45:26.240 to the Lambda logs. So say you set up everything on EC2, it's not going to pump everything that's 00:45:26.240 --> 00:45:32.160 on EC2 automatically to those logs, right? You'd have to specify it. It pumps some base stuff. 00:45:32.380 --> 00:45:42.160 But with an EC2 or sorry, an ECS task, it'll pump everything that's getting that you'd see normally if you ran it from Docker, you know, Docker and same thing with Lambdas. 00:45:42.330 --> 00:45:51.980 And so just as long as you're not telling some process to write to disk and some log file, which can happen, you know, then you'd have to tell it to pipe that out to CloudWatch logs. 00:45:52.400 --> 00:46:03.040 So you're saying like if I basically did Docker or Docker Compose Logs-F type of thing where it streams out whatever's coming out of the container, it'll just send that over to Watch? 00:46:03.340 --> 00:46:05.840 And those are all by default, super easy to set up. 00:46:06.000 --> 00:46:12.160 Just the only problem would be if you said, hey, application layer, write this secondary log to this file. 00:46:12.550 --> 00:46:14.180 And that wasn't getting piped out somehow. 00:46:14.520 --> 00:46:20.660 I've successfully had it write to a file separately and then told the Docker process to pipe that out to be a tail. 00:46:20.680 --> 00:46:22.640 That was the end of the Docker run command was tail. 00:46:22.780 --> 00:46:24.960 It's a little sloppy, but you can do it. 00:46:24.980 --> 00:46:29.200 A better way to do it would be to use AWS CLI commands 00:46:29.460 --> 00:46:34.560 or some of the AWS SDK to pump it to a CloudWatch log stream. 00:46:35.180 --> 00:46:36.660 And then you'd have that ready to go. 00:46:36.660 --> 00:46:37.740 And a lot of my clients do that. 00:46:38.120 --> 00:46:39.360 It's a little cleaner than the first one. 00:46:39.460 --> 00:46:39.640 Sure. 00:46:40.060 --> 00:46:42.700 Another option might be set up two destinations for the logging 00:46:42.820 --> 00:46:45.900 because you can set multiple, you could say this file and standard out 00:46:46.100 --> 00:46:47.040 and just let it rip. 00:46:47.180 --> 00:46:49.380 And then just that becomes part of the Docker logs. 00:46:49.900 --> 00:47:19.840 Yeah, exactly. So just, you know, it's pretty simple once you get in there. Talked about insight. But yeah, the metrics. Another thing about metrics is you can set alarms, not just to wake you up if something bad's happening, but also to trigger auto scaling, which is really nice. And so you could scale up and down based on not just. We had something. We were pulling from queues. We had a worker. It was not serverless. It was provisioned. And we had a worker. And when the queue got too high, you know, got too big, we wanted it to boot up another one. 00:47:19.900 --> 00:47:24.600 one but that's normally if you just use a default one it's just like cpu or memory or something like 00:47:24.740 --> 00:47:30.280 that but we were able to configure it so it would pull from the count of messages in flight and use 00:47:30.340 --> 00:47:34.640 that to auto scale up and down and that was really convenient that wasn't that was a client request 00:47:34.860 --> 00:47:39.320 for that one and that's cool yeah it seems like it makes way more sense than just cpu yeah because 00:47:39.420 --> 00:47:44.180 maybe you've got some process that spin off like an analytics thing pin in the cpu but it's not 00:47:44.320 --> 00:47:48.460 really that busy right and scaling won't make a difference but if you've got something like maybe 00:47:48.480 --> 00:47:50.540 average response time if it gets too big. 00:47:50.990 --> 00:47:51.780 Just scale up. 00:47:52.120 --> 00:47:54.560 If it's over 300 milliseconds on average or median, 00:47:54.930 --> 00:47:55.560 we just need more. 00:47:56.010 --> 00:47:59.680 Yeah, and you can pop all that stuff into your chat of choice. 00:47:59.920 --> 00:48:00.980 We've got all that for my clients. 00:48:01.030 --> 00:48:03.720 It goes right into a channel and on-call people jump on it. 00:48:03.860 --> 00:48:05.100 I don't do on-call work anymore. 00:48:05.790 --> 00:48:08.240 I just set it up so I train the new guys to do it. 00:48:08.460 --> 00:48:09.580 Yeah, that's the way to do it. 00:48:09.650 --> 00:48:11.300 Tell me in the morning what you did, what went wrong. 00:48:12.700 --> 00:48:13.300 Yeah, excellent. 00:48:13.610 --> 00:48:15.040 Okay, so that's logs. 00:48:15.320 --> 00:48:16.920 I feel like people probably got this. 00:48:17.260 --> 00:48:22.040 don't hand deploy, but also a little bit of a, well, then what? You know, CI/CD, right? 00:48:22.500 --> 00:48:27.380 Yep. So I'm going to go take a time machine back about 20 years, a long time ago, we would 00:48:27.660 --> 00:48:32.720 basically FTP up, you know, stuff from, from our computers to a server or something like that. 00:48:32.810 --> 00:48:36.940 And then it would run. So if you didn't, depending on how your deployment was, 00:48:37.420 --> 00:48:41.580 kind of have to hand go through it. Well, nowadays people want to be moving super fast. 00:48:41.670 --> 00:48:45.800 So we've created

these systems that you just commit it gets approved from the pull request 00:48:45.820 --> 00:48:50.400 from whomever. And then we rebuild the image like we talked about. If it's Docker, we rebuild the 00:48:50.580 --> 00:48:56.760 image. If it's Lambda, we build whatever and upload the file as we need. But you also can do database 00:48:57.020 --> 00:49:03.140 migrations as well. I've got clients that use MySQL and their table updates all run through there. 00:49:03.430 --> 00:49:09.060 So you can have that all run there. You can have automated tests, test-driven development. If that 00:49:09.320 --> 00:49:12.860 is something that suits your need, I'd encourage it for anybody that's big enough and has something 00:49:12.880 --> 00:49:13.740 They don't want to go down. 00:49:14.140 --> 00:49:16.120 And if you're just starting, I'm a little looser on it. 00:49:16.220 --> 00:49:19.400 But having those build pipelines in place saves you a ton of time. 00:49:20.339 --> 00:49:23.100 And of course, if they fail, have them send you a message. 00:49:24.380 --> 00:49:26.540 So do you recommend CodeCommit from AWS? 00:49:27.440 --> 00:49:30.520 No, I've actually, a long time ago, I had my clients migrate off CodeCommit. 00:49:30.540 --> 00:49:32.420 Is CodeCommit even running? 00:49:32.760 --> 00:49:34.860 CodeCommit is different than CodeBuild and CodePipeline. 00:49:34.940 --> 00:49:35.720 It was their GitHub. 00:49:36.759 --> 00:49:38.200 Oh, I think it might be shutting down. 00:49:38.260 --> 00:49:39.140 Okay, sorry, I got this backwards. 00:49:39.520 --> 00:49:41.280 Yeah, so CodePipeline and CodeBuild. 00:49:41.820 --> 00:49:43.420 So pipeline is more of an orchestration layer. 00:49:43.760 --> 00:49:43.860 Okay. 00:49:43.980 --> 00:49:48.340 It listens for the commit and then you can tell it to do a bunch of stuff, upload S3, 00:49:48.750 --> 00:49:50.320 you know, or deploy or whatever. 00:49:50.620 --> 00:49:53.400 Code build is actually kind of ECS. 00:49:53.540 --> 00:49:56.420 It's like, it spins up a Docker container or a Lambda now. 00:49:56.540 --> 00:49:57.340 That's actually kind of new. 00:49:57.800 --> 00:49:59.360 And that you can have that Docker container. 00:49:59.860 --> 00:50:02.380 That's just a build container, build your application container. 00:50:02.560 --> 00:50:06.140 It's a little, it sounds a little weird, but basically you could just have it run your Docker 00:50:06.320 --> 00:50:09.040 build in the cloud, you know, and it'll just do it. 00:50:09.040 --> 00:50:11.340 And you say Docker push and pushes it out to ECR. 00:50:11.640 --> 00:50:16.980 the elastic container repo that's basically where the images are stored for docker and then then the 00:50:17.080 --> 00:50:22.300 next step that it'll throw it back to code pipeline you sell code pipeline do a red green deploy do a 00:50:22.300 --> 00:50:25.960 regular deploy do whatever interesting i never really thought about it but it makes perfect 00:50:26.120 --> 00:50:30.540 sense that they wouldn't be going docker pull from docker hub why do you leave their cloud right 00:50:30.720 --> 00:50:35.860 well and you can but they've also created this and you've got your granular iam permissions like 00:50:35.860 --> 00:50:40.140 we talked about before iam roles that you can now super secure without having to have two different 00:50:40.160 --> 00:50:44.660 things in the mix. They do that a lot. They take whatever's working out there and they just rebuild 00:50:44.820 --> 00:50:48.280 it. And that's probably why they have billion services at this point. Exactly. That's why the 00:50:48.420 --> 00:50:51.540 console looks like it does. All right. Now we've got to pick up some speed. We're still on our, 00:50:52.160 --> 00:50:55.420 on a thing. So maybe give us a little bit of a lightning round in the last couple here. 00:50:55.650 --> 00:50:59.140 Okay. I knew I put it in here. I was looking at my notes and I've got it in small text here. Yes. 00:50:59.460 --> 00:51:03.620 So SSM session manager. Okay. So like we talked about before, so if you boot up an ECS, 00:51:04.940 --> 00:51:08.880 sorry, EC2, EC2, that's the old school virtual machines, the big chunky ones that people like 00:51:08.900 --> 00:51:12.980 to nurse, you have the option to put an SSH key on there. 00:51:13.320 --> 00:51:16.440 So that way you can get into it and then do all that hand tuning you're doing. 00:51:17.360 --> 00:51:18.700 Again, I kind of touched on this. 00:51:18.800 --> 00:51:19.620 We'll go fast on this one. 00:51:20.020 --> 00:51:22.460 Use SSM session manager instead of the bastion. 00:51:23.220 --> 00:51:25.640 If you can, it'll make it a lot easier for you. 00:51:25.760 --> 00:51:27.840 So now we just caught some time up there because I already covered it. 00:51:27.880 --> 00:51:28.280 Yeah, perfect. 00:51:28.480 --> 00:51:28.880 Yeah, I love it. 00:51:29.060 --> 00:51:29.680 Okay, this is cool. 00:51:29.800 --> 00:51:30.660 That's really good advice. 00:51:31.100 --> 00:51:33.380 I mean, S3 public, why not? 00:51:33.900 --> 00:51:37.140 This one, I have seen so many clients stunned by it. 00:51:37.160 --> 00:51:43.700 I did an assessment, which is like my entry, like the first thing I do with people a lot of times is a short-term assessment to make, 00:51:43.800 --> 00:51:45.980 to give them an idea where they're at, and I map out the whole system. 00:51:46.779 --> 00:51:50.800 And they were just telling me, they were like, we really can't have these files public. 00:51:51.080 --> 00:51:52.660 They're binary files. 00:51:52.760 --> 00:51:56.100 Like, they would be big trouble for us if they were, you know, if anybody could hack it, could you check that out? 00:51:56.400 --> 00:51:59.600 And sure enough, the bucket was public to anybody on the planet. 00:51:59.740 --> 00:52:00.780 They just grabbed it out of there. 00:52:00.900 --> 00:52:04.700 It would have been tough to crawl it because they don't have listing publicly,

but it would have, could have, 00:52:04.860 --> 00:52:07.660 The pattern wasn't so bad that you couldn't have had something brute force. 00:52:07.770 --> 00:52:09.300 You could like enumerate, attack it. 00:52:09.400 --> 00:52:09.520 Yeah. 00:52:09.900 --> 00:52:10.040 Yeah. 00:52:10.210 --> 00:52:12.760 So S3 is a wonderful binary storage. 00:52:12.870 --> 00:52:13.880 It is not a CDN. 00:52:14.090 --> 00:52:19.380 It is not cost effective to serve up things at mass very fast through that. 00:52:19.490 --> 00:52:21.680 So you want to use the right tools for the job. 00:52:22.020 --> 00:52:23.920 And it's also secure-ish. 00:52:23.990 --> 00:52:26.500 But what you can do is you can do cloud front signed URLs. 00:52:27.440 --> 00:52:30.240 So you could say you could be very granular with your permissions. 00:52:30.820 --> 00:52:36.480 this like you said with the service earlier i could say this url is valid for exactly this 00:52:36.510 --> 00:52:40.960 amount of time and no longer and it goes to exactly this one file and no other files and 00:52:40.960 --> 00:52:44.480 i can send that out there you can do the same thing with uploads actually the uploads would go 00:52:44.780 --> 00:52:50.900 quickly to s3 but this you signed uploads so that way you're very that you've got this url lasts for 00:52:51.220 --> 00:52:55.400 five minutes you know two minutes 30 seconds whatever and you can get the upload done that 00:52:55.380 --> 00:53:00.480 that way so just i'm going to reiterate signed uploads with s3 make the bucket private use a 00:53:00.510 --> 00:53:06.620 signed url and then sign um use cloudfront and then if you need to lock it down use signed urls 00:53:06.620 --> 00:53:12.520 on the cloudfront side um but cloudfront is going to be so much more cost effective to serve up a 00:53:12.700 --> 00:53:17.460 lot high volume of binary or you know any any assets javascript right it's optimized for it yeah 00:53:17.760 --> 00:53:24.340 yeah so just um definition wise cloudfront they're cdn yeah that's right okay cost do people care 00:53:24.360 --> 00:53:29.820 about cost aws i don't know is that a thing uh so i i've been doing an experiment proactive 00:53:30.060 --> 00:53:33.860 engagement where i've got agents crawling around the internet uh looking for people complaining 00:53:34.240 --> 00:53:37.840 and cost is pretty much the number one thing why the heck did this cost me this much why the heck 00:53:37.840 --> 00:53:42.360 did it cost me that much you know and if you don't understand the costs it's really it's ugly but 00:53:42.480 --> 00:53:47.520 learn to use cost explorer because they're going to have a new service that comes out next we use 00:53:47.700 --> 00:53:51.160 there i wrote about this in march march they released a new charge if you didn't upgrade 00:53:51.180 --> 00:53:56.060 from older database versions they had a past end of life fee so all of a sudden my clients were 00:53:56.140 --> 00:54:00.140 getting smacked with about four grand a month and end of life fees and you're just like huh that 00:54:00.160 --> 00:54:04.680 where did that come from so but we didn't know that's terrible they're not even using it it's 00:54:04.680 --> 00:54:09.360 good yeah well they were using the database they just hadn't they should they should have moved 00:54:09.540 --> 00:54:14.100 up they should yeah yeah on IWS's side it's expensive to keep that older stuff running so 00:54:14.340 --> 00:54:18.100 it kind of makes sense but I wish they would have given us a better heads up but either way so I we 00:54:18.120 --> 00:54:22.660 go in there in the cost explorer and i was able to find that exact one it's really if you know what 00:54:22.670 --> 00:54:26.820 you're doing and you know how to switch between services and usage types and breakdowns you can 00:54:27.080 --> 00:54:31.420 find exactly where that jump is where it came from and then you can usually track that back to a very 00:54:31.600 --> 00:54:35.900 specific resource um and it also allows you to do tagging which is great you can tag production you 00:54:35.900 --> 00:54:40.720 can tag staging you can tag microservice x whatever you know so you can tag them and you can say look 00:54:40.920 --> 00:54:45.480 qa is actually costing us twenty thousand dollars a month can we just if you had hardware tagged as 00:54:45.500 --> 00:54:46.000 QA, yeah. 00:54:46.380 --> 00:54:46.440 Nice. 00:54:46.730 --> 00:54:48.020 Just to kind of bring this home here, 00:54:48.130 --> 00:54:51.680 and it's a beautiful loop back in a terrible or morbid way, 00:54:51.820 --> 00:54:54.980 but there was this story of this woman who created this, 00:54:55.210 --> 00:54:58.220 Cara, it's the app, but she created this. 00:54:58.410 --> 00:55:02.260 She really disliked AI-generated art because she was an artist, 00:55:02.400 --> 00:55:05.100 so she created this app that would use AI to tell 00:55:05.280 --> 00:55:07.280 if a piece of art was legit or AI. 00:55:07.880 --> 00:55:10.600 And she's coolified to host it at Vercel. 00:55:10.820 --> 00:55:14.680 Got a $95,000 Vercel bill in like a couple of days. 00:55:15.100 --> 00:55:18.600 As a student or just an independent artist or something like that, it was really not good. 00:55:18.940 --> 00:55:21.860 Yeah, that's not good. 00:55:22.220 --> 00:55:25.140 I mean, I don't want to down on anybody, but that's probably a token cost. 00:55:25.200 --> 00:55:28.360 And as we're finding out, the costs can be prohibitive. 00:55:28.820 --> 00:55:30.420 I think it was just CPU, actually. 00:55:30.900 --> 00:55:32.860 I don't think it actually was. 00:55:33.400 --> 00:55:37.620 It became like number one on the App Store or something like that. 00:55:37.940 --> 00:55:41.660 And Vercel had sent her a warning like, hey, your bill is blowing up. 00:55:41.680 --> 00:55:43.760 You need to either approve or disapprove. 00:55:44.020 --> 00:55:47.080 there was no upper bounds on cost set at all. 00:55:47.400 -->

00:55:49.420 And she didn't respond to the warning emails 00:55:49.800 --> 00:55:51.140 for like three or four days or something like that. 00:55:51.160 --> 00:55:53.320 It was like a cascading set of lessons 00:55:53.520 --> 00:55:54.380 that should be taken from this. 00:55:54.680 --> 00:55:54.900 Wow. 00:55:55.539 --> 00:55:57.720 Yeah, that's, I mean, definitely set up cost alerts. 00:55:58.060 --> 00:55:58.580 Keep an eye on it. 00:55:59.420 --> 00:56:00.520 You can set up budgets in AWS. 00:56:01.420 --> 00:56:03.320 And I mean, I, every week, 00:56:03.600 --> 00:56:05.180 so I have office hours with my clients. 00:56:05.260 --> 00:56:06.260 It's like an advisory session 00:56:06.400 --> 00:56:08.080 and I kick it off typically budget sheet 00:56:08.200 --> 00:56:08.780 right in front of me. 00:56:09.080 --> 00:56:10.620 So I was like, okay, here we're at, 00:56:10.680 --> 00:56:12.100 we saw this spike, what happened there? 00:56:12.240 --> 00:56:12.820 Oh, you know. 00:56:13.060 --> 00:56:14.320 Sometimes spikes are amazing. 00:56:14.510 --> 00:56:17.420 I mean, that means people are using your stuff, but sometimes they're not. 00:56:17.720 --> 00:56:17.840 Yeah. 00:56:18.100 --> 00:56:18.380 All right. 00:56:18.780 --> 00:56:21.020 You know, another time that they're not is when you're getting DDoSed. 00:56:21.300 --> 00:56:22.800 Yeah, this brings it right in. 00:56:22.940 --> 00:56:24.180 So DDoSs are everywhere. 00:56:24.400 --> 00:56:25.640 Cyber attacks all the time. 00:56:25.690 --> 00:56:29.020 The bigger you are, once you get to that big league, they're coming for you from the front. 00:56:29.120 --> 00:56:32.280 They're coming for phishing attacks from the side, from the back, from everything you can. 00:56:33.280 --> 00:56:36.300 The way we deal with that on AWS, there's a couple different ways. 00:56:37.279 --> 00:56:39.640 WAF is my tool of choice typically. 00:56:39.960 --> 00:56:41.160 There's some that you can, 00:56:41.520 --> 00:56:42.180 they've got some tools 00:56:42.310 --> 00:56:44.260 where they do some really fancy analytical stuff 00:56:44.819 --> 00:56:45.760 to block stuff. 00:56:46.700 --> 00:56:48.220 And that can work, but at volume, 00:56:48.640 --> 00:56:51.120 it's not really the most cost effective. 00:56:51.360 --> 00:56:51.860 You know, if you're, 00:56:51.980 --> 00:56:53.020 every request that comes through, 00:56:53.140 --> 00:56:54.280 you're running through some AI model, 00:56:54.400 --> 00:56:55.320 it's going to add up real quick. 00:56:55.820 --> 00:56:57.560 But there's a lot of stuff you can do that's obvious. 00:56:57.690 --> 00:56:59.940 Like, okay, are they looking for the WordPress admin? 00:57:00.320 --> 00:57:00.960 And you're not running WordPress. 00:57:01.000 --> 00:57:02.720 That should just be an absolute flag. 00:57:02.980 --> 00:57:05.140 Like if it's slash WP, you know, 00:57:05.500 --> 00:57:07.880 WP admin.php, Instaban is what it should be. 00:57:08.140 --> 00:57:08.800 Yep,

you got it. 00:57:08.900 --> 00:57:12.060 You can do rate limiting as well, you know, and then you do rate limiting by route. 00:57:12.260 --> 00:57:13.380 We just did that for a client. 00:57:14.980 --> 00:57:17.780 There's a lot of different, yeah, there's a lot of different fun stuff you can do there. 00:57:17.780 --> 00:57:20.440 And they've got some of those things already kind of packaged in there. 00:57:20.720 --> 00:57:25.100 So you don't always need to be using like the smartest, like, you know, an AI model to evaluate. 00:57:25.260 --> 00:57:25.800 It's real simple. 00:57:25.900 --> 00:57:30.880 If the URL is, you know, something in the well-known thing that shouldn't be there, then you know it. 00:57:32.000 --> 00:57:32.860 Sorry, I'm getting sidetracked. 00:57:33.060 --> 00:57:34.580 You can also do challenges. 00:57:34.880 --> 00:57:40.660 So that's a JavaScript challenge that runs beneath the scene, like in your browser, and the user won't even really notice it. 00:57:40.890 --> 00:57:43.140 Or you can do a captcha if you really want to get into it. 00:57:43.600 --> 00:57:44.480 Captchas are a bit more expensive. 00:57:44.870 --> 00:57:46.400 Day challenges are much more cost effective. 00:57:47.140 --> 00:57:48.380 But there's a time and a place for them. 00:57:48.380 --> 00:57:51.600 You just got to kind of phenomps your way to how much you want to spend on it. 00:57:51.720 --> 00:58:01.180 Yeah, I've found that turnstile flair has gone a long ways towards keeping the bots away at the same time not being, you know, I just can't take another find the fire hydrant. 00:58:01.210 --> 00:58:01.740 I can't do it. 00:58:02.780 --> 00:58:04.520 You know, it's like, find all the fire hydrants like I did. 00:58:04.800 --> 00:58:06.640 And then one will fade in later. 00:58:06.820 --> 00:58:08.160 And it'll say, well, you didn't find all of them. 00:58:08.220 --> 00:58:09.500 Now you're banned because you didn't find them. 00:58:09.620 --> 00:58:11.120 So now you're going to do like fire hydrants. 00:58:11.140 --> 00:58:11.680 I can't take it. 00:58:11.740 --> 00:58:12.500 Or bicycles, you know. 00:58:12.700 --> 00:58:14.640 Yeah, well, that's where the challenges come into play. 00:58:15.300 --> 00:58:20.020 If they're using something like API-FI, which is if I'm trying to crawl something at scale, 00:58:20.240 --> 00:58:21.460 that's my tool to go to. 00:58:21.560 --> 00:58:22.700 It's not going to find that. 00:58:23.600 --> 00:58:25.020 But, you know, it'll do pretty good. 00:58:25.120 --> 00:58:27.180 You also can block entire countries, it turns out. 00:58:27.280 --> 00:58:29.760 I've got clients that just decided to block France once. 00:58:29.980 --> 00:58:30.500 I'm like, why? 00:58:30.780 --> 00:58:32.320 We can figure out this traffic. 00:58:32.740 --> 00:58:35.160 They beat their team in the World Cup and that's it. 00:58:35.180 --> 00:58:36.700 No more access for France. 00:58:36.920 --> 00:58:38.240 Sorry, our French friends. 00:58:38.960 --> 00:58:39.580 It's crazy. 00:58:40.020 --> 00:58:40.180 All right. 00:58:40.220 --> 00:58:44.100 Well, let's spend a few minutes as we kind of wind down the show. 00:58:44.220 --> 00:58:45.520 Now people got the idea. 00:58:45.940 --> 00:58:48.080 I mean, nobody's calm at 3 a.m. 00:58:48.080 --> 00:58:49.200 I think this is true. 00:58:49.820 --> 00:58:53.900 I've gone through the experience of multiple times, like your website's down. 00:58:54.120 --> 00:58:54.840 Oh, no, surely not. 00:58:54.940 --> 00:58:55.720 Oh, my gosh. 00:58:55.800 --> 00:58:56.580 And then why? 00:58:56.760 --> 00:58:58.240 And then there's just all this hack. 00:58:58.440 --> 00:58:59.240 Did I do something wrong? 00:58:59.840 --> 00:59:01.680 It's never fun when you're in it. 00:59:01.960 --> 00:59:03.940 And that's kind of your Cloud War Games thing, right? 00:59:04.100 --> 00:59:05.100 To help people through that. 00:59:05.420 --> 00:59:11.860 Yeah, you'd rather learn, you know, get the stress inoculation during a simulation, a 45-minute game, 00:59:12.300 --> 00:59:17.560 than at 3 a.m., you know, or when the entire board of directors is standing behind you 00:59:17.900 --> 00:59:19.420 because you're the guy that's supposed to fix it. 00:59:19.720 --> 00:59:23.920 So I saw, I mean, many times I saw it and I would be working with juniors and I'd try and, 00:59:24.599 --> 00:59:28.920 I'd be, you know, doing my, in a real incident, you know, trying to solve the problem. 00:59:29.130 --> 00:59:31.640 I'd pass something out to them, you know, so, okay, you look at this. 00:59:32.319 --> 00:59:36.280 trying to get them to learn and they just you know they'd freeze up and just like i don't know and 00:59:36.280 --> 00:59:40.100 then you'd ask them later like hey what was the problem you know you do that now so i mean another 00:59:40.200 --> 00:59:43.900 part of the war games is afterwards we always post-mortem and if you ever have an outage and 00:59:44.140 --> 00:59:48.780 especially if you got people freezing up you should post-mortems every time uh that's the way you learn 00:59:49.120 --> 00:59:53.140 and that's actually where i got most of the content for things i run people through on cloud war games 00:59:53.300 --> 01:00:00.500 from things i've actually suffered through so this is a program i can go sign up for and is it like a 01:00:00.500 --> 01:00:06.520 paid course or a community or what's the experience no so so it's it's actually evolving right now 01:00:06.650 --> 01:00:13.240 um we were running it where i did monthly ones and just the scheduling was complex so we're we're 01:00:13.280 --> 01:00:18.280 adapting a little bit um the you can sign up and you can get access to a bunch of what i call 01:00:18.540 --> 01:00:22.060 asynchronous challenges for free completely free there's just we'll send you one every week for 01:00:22.330 --> 01:00:28.060 i don't remember how long it goes um but the thing where we're trying to move into is trying

to 01:00:28.020 --> 01:00:33.360 move it into, the Schematical has got a community offering basically where we, it's kind 01:00:33.360 --> 01:00:36.340 of more like coaching and a little bit more ongoing. So we're trying to move it in there 01:00:36.740 --> 01:00:40.760 where we can have it. I don't know, trying, I'm trying to get people in discord so we can 01:00:41.060 --> 01:00:44.940 plan these things a little bit better. It's a bit, you know, email list is great, but there's 01:00:44.940 --> 01:00:49.020 not like an organic, like, Hey, let's run a game right now. You know? Yeah. Discord is nice. Yeah. 01:00:49.400 --> 01:00:54.100 Yeah. And I'm also working this in my, we, we brought this into my consulting as well for the 01:00:54.120 --> 01:01:00.040 big companies is I do this for training, but also it's a great way to figure out, especially with AI 01:01:00.559 --> 01:01:03.880 resumes and all that stuff. It's a great way to figure out if someone actually knows their stuff. 01:01:04.070 --> 01:01:07.580 And even if they do know their stuff, are they a team player? So you can throw, you know, 01:01:07.950 --> 01:01:13.180 a couple of your applicants in there, candidates, and you'll see if one of them's my comics, 01:01:13.250 --> 01:01:17.100 the lone wolf programmer who doesn't talk to anybody and just like goes heads down and, 01:01:17.440 --> 01:01:19.800 or are they a team player? And they're like communicating, they're shouting out, 01:01:19.890 --> 01:01:23.240 Hey, you try this. And that those soft skills. I didn't know this when I first started it, 01:01:23.260 --> 01:01:27.560 but it really lets you find the people that are soft skills that you want being a leader of a team. 01:01:27.810 --> 01:01:31.020 And so we're working with bigger companies to run that, 01:01:31.660 --> 01:01:34.380 both to train their people internally, but also as a recruiting tool. 01:01:34.800 --> 01:01:36.480 Yeah, that's super neat. I like it. 01:01:37.180 --> 01:01:39.120 Thinking back of why did I freak out, 01:01:39.830 --> 01:01:43.220 a lot of times it's because I did something a couple of years ago 01:01:43.480 --> 01:01:47.080 and I had forgotten how I did it and even where the thing I needed was. 01:01:47.520 --> 01:01:53.140 Like one of the most annoying ones I remember is the SSH or just the TLS certificate. 01:01:53.200 --> 01:01:56.900 I was using for MongoDB to encrypt expired. 01:01:57.440 --> 01:01:57.760 I'm like, why? 01:01:58.180 --> 01:01:58.800 First of all, I'm like, 01:01:58.940 --> 01:02:00.840 why did I set it to be a one year expiry 01:02:01.020 --> 01:02:01.880 for an internal cert? 01:02:02.140 --> 01:02:03.240 I really should have made it longer 01:02:03.520 --> 01:02:04.800 or I should have written down what I did. 01:02:05.020 --> 01:02:06.040 But then I couldn't even remember, 01:02:06.380 --> 01:02:07.960 like, well, what was the command I ran? 01:02:08.180 --> 01:02:08.780 And like, how just, 01:02:09.080 --> 01:02:10.440 so I was just running around, 01:02:10.760 --> 01:02:13.240 like checking all these old logs and projects. 01:02:13.600 --> 01:02:14.140 Where is it? 01:02:14.480 --> 01:02:16.660 What are the things that you see 01:02:16.820 --> 01:02:18.740 that drive people crazy that they could maybe, 01:02:19.160 --> 01:02:20.400 what are some of the learnings, I guess, 01:02:20.440 --> 01:02:21.020 you're getting out of this? 01:02:21.080 --> 01:02:26.200 Like for me, it would be like, write the stuff down in a consistent place so I can get to it calmly. 01:02:26.560 --> 01:02:34.600 Right. Well, and so I hate to say some people don't like it, but a problem you just had there would be well fixed by a managed service of some type. 01:02:34.820 --> 01:02:37.120 AWS is KMS is a certificate manager. 01:02:37.580 --> 01:02:42.280 You know, these days in the age, you know, it saves you how many, how much money would you have paid that night? 01:02:42.620 --> 01:02:46.480 Or did it cost the company that now you could just pay a AWS KMS? 01:02:46.720 --> 01:02:50.620 You know, that's it would have saved you probably twice as much or a lot more than you spent that night. 01:02:50.700 --> 01:02:57.140 headaches you know yeah yeah so um i'd have more hair at least um yeah as far as the common ones 01:02:57.380 --> 01:03:04.500 it's the most common ones i see is misconfigured im you know misconfigured bpc public s3 happens 01:03:05.040 --> 01:03:08.840 oh too much i can i can't do an audit without finding something in s3 that's not supposed to 01:03:08.840 --> 01:03:12.620 be public you know sales numbers something like that you don't want your competitor to have so 01:03:13.090 --> 01:03:18.540 those are the most common ddos is like i said every all the time um but then it's the it's the one 01:03:18.560 --> 01:03:22.720 off. There's one I do. I don't want to disclose it because I love it, but there's a ransomware 01:03:22.880 --> 01:03:27.800 attack. It's my advanced one. I call it the Kobayashi Maru because it's pretty brutal. 01:03:28.640 --> 01:03:33.380 Luckily, I've never had a client hit with it, but it is a brutal attack that's specific to AWS, 01:03:33.500 --> 01:03:38.020 and it can be run one command if you've got the right privileges, and it's brutal. 01:03:38.260 --> 01:03:44.080 That's not good. That's not good. I saw that you've got a couple of these live events on video, 01:03:44.360 --> 01:03:47.280 like a beginner one and a more advanced one. 01:03:47.440 --> 01:03:49.500 So maybe I'll link to these for people in the show notes. 01:03:49.800 --> 01:03:50.160 Sounds good. 01:03:50.180 --> 01:03:52.660 And they're on youtube.com slash grammatical if you need it. 01:03:53.440 --> 01:03:53.760 All right. 01:03:54.000 --> 01:03:55.660 Well, maybe bring it home to people. 01:03:56.060 --> 01:03:56.500 Sounds good. 01:03:56.840 --> 01:03:57.440 Closing advice. 01:03:57.440 --> 01:03:59.340 Since we've been talking, I came up with like three

more. 01:03:59.660 --> 01:04:02.380 So if anybody wants my other ones, you know, reach out. 01:04:02.440 --> 01:04:02.920 Want a lightning round? 01:04:03.020 --> 01:04:04.640 Let's want a lightning round them real quick. 01:04:04.880 --> 01:04:05.080 Sure. 01:04:05.240 --> 01:04:05.660 Well, yeah. 01:04:06.000 --> 01:04:09.660 So AI, SageMaker for your Python crew, SageMaker. 01:04:10.000 --> 01:04:11.080 Let's check that service out. 01:04:11.500 --> 01:04:16.620 It's if you want the GPUs, have more granular control than you can on basically any other service. 01:04:17.080 --> 01:04:19.200 And it's Python first, data science first. 01:04:19.360 --> 01:04:22.920 So if you're data science, you know, you like it's check out SageMaker. 01:04:23.720 --> 01:04:27.340 If you're not that stuff, Bedrock or if you just generic stuff, Bedrock's amazing as well. 01:04:27.460 --> 01:04:29.900 It's their Netflix for AI. Phenomenal. I'd check that out. 01:04:30.280 --> 01:04:32.260 The other one, the Data Lake Stack I was talking about earlier. 01:04:32.400 --> 01:04:39.900 I need to do some Terraform scripts for this, but they have I didn't talk about adventure in architecture, but great cues that you can use. 01:04:40.680 --> 01:04:44.640 Kinesis is a Firehose queue, meaning that you can have a lot of different things consuming from it. 01:04:44.740 --> 01:04:47.280 So if you've got 10 different worker services, they all can consume from it. 01:04:47.280 --> 01:04:47.500 It's great. 01:04:48.160 --> 01:04:48.540 Pump that. 01:04:48.900 --> 01:04:55.100 So whenever your application layer fires off an event, hey, this was updated, a user was created, a sale was made, pump it into Kinesis. 01:04:55.480 --> 01:04:59.940 And then you can have Firehose save that as a parquet file to S3. 01:05:00.140 --> 01:05:00.960 It's extremely cheap. 01:05:01.000 --> 01:05:03.560 It doesn't cost fractions of a penny. 01:05:03.600 --> 01:05:04.120 It's so cheap. 01:05:04.280 --> 01:05:09.700 There's super interesting Python libraries that you can mount S3 parquet files and stuff and do query. 01:05:09.860 --> 01:05:12.420 There's really a lot of cool integrations with Python and Parquet, yeah. 01:05:12.720 --> 01:05:15.620 Yeah, and you can query it with Glue, 01:05:15.900 --> 01:05:18.320 which I believe has some Python in there. 01:05:18.600 --> 01:05:20.440 But if you didn't want to do that, you're just an SQL guy, 01:05:20.760 --> 01:05:22.680 Athena can talk and query to it as well. 01:05:23.040 --> 01:05:24.580 So I need to do some writing down on this, 01:05:24.660 --> 01:05:26.720 but it's a phenomenal data-like thing. 01:05:27.060 --> 01:05:30.880 Don't try and keep sales from 2003 in your production database. 01:05:31.320 --> 01:05:33.920 Move all that stuff, that old stuff, off of it. 01:05:33.960 --> 01:05:36.480 Keep your production database nice and lean and moving fast. 01:05:36.820 --> 01:05:37.040 Awesome. 01:05:37.480 --> 01:05:38.960 Well, Matt, thank you for being here. 01:05:39.160 --> 01:05:39.780 It's been really awesome. 01:05:40.200 --> 01:05:42.020 I'll put all your contact info in the show notes 01:05:42.100 --> 01:05:43.920 for people who want to get in touch. 01:05:44.100 --> 01:05:46.280 And yeah, thanks for keeping our clouds a little healthier. 01:05:46.600 --> 01:05:47.380 Yeah, sounds good. 01:05:47.700 --> 01:05:48.020 Happy to help. 01:05:48.400 --> 01:05:49.360 Feel free to reach out to me. 01:05:49.520 --> 01:05:51.140 And, you know, thanks for having me. 01:05:51.460 --> 01:05:51.880 Yeah, you bet. 01:05:52.000 --> 01:05:52.100 Bye-bye. 01:05:52.860 --> 01:05:55.000 This has been another episode of Talk Python To Me. 01:05:55.300 --> 01:05:56.080 Thank you to our sponsors. 01:05:56.400 --> 01:05:57.580 Be sure to check out what they're offering. 01:05:57.820 --> 01:05:59.160 It really helps support the show. 01:05:59.640 --> 01:06:01.000 Take some stress out of your life. 01:06:01.340 --> 01:06:04.560 Get notified immediately about errors and performance issues 01:06:04.760 --> 01:06:06.780 in your web or mobile applications with Sentry. 01:06:07.340 --> 01:06:11.700 Just visit talkpython.fm/century and get started for free. 01:06:12.320 --> 01:06:14.700 Be sure to use our code talkpython26. 01:06:15.680 --> 01:06:19.040 That's talkpython, the numbers two, six, all one word. 01:06:19.540 --> 01:06:22.400 And it's also brought to you by Talk Python Courses. 01:06:22.980 --> 01:06:25.240 Course completion certificates are now live. 01:06:25.360 --> 01:06:29.660 If you finished a course, there's a certificate waiting for you on your account page right now. 01:06:30.160 --> 01:06:35.880 Download it as a PDF or add it to your LinkedIn profile with one click under licenses and certifications. 01:06:36.440 --> 01:06:37.760 Same section as your formal degrees. 01:06:38.940 --> 01:06:42.760 Visit training.talkpython.fm/account to see what you've already earned. 01:06:43.720 --> 01:06:45.560 If you or your team needs to learn Python, 01:06:45.780 --> 01:06:49.780 we have over 270 hours of beginner and advanced courses on topics 01:06:49.940 --> 01:06:55.840 ranging from complete beginners to async code, Flask, Django, HTML, and even LLMs. 01:06:56.040 --> 01:06:58.440 Best of all, there's no subscription in sight. 01:06:58.900 --> 01:07:00.660 Browse the catalog at talkpython.fm. 01:07:01.400 --> 01:07:04.720 And if you're not already subscribed to the show on your favorite podcast player, 01:07:05.360 --> 01:07:06.020 what are you waiting for? 01:07:06.700 --> 01:07:08.440 just search for Python in your podcast player 01:07:08.550 --> 01:07:09.420 we should be right at the top 01:07:09.830 --> 01:07:11.400 if you enjoyed that geeky rap song 01:07:11.490 --> 01:07:12.660 you can download the full track 01:07:12.830 --> 01:07:14.740 the link is actually in your podcast blur show notes 01:07:15.340 --> 01:07:16.900 this is your host Michael Kennedy 01:07:17.300 --> 01:07:18.360 thank you so much for listening 01:07:18.550 --> 01:07:19.340 I really appreciate it 01:07:19.770 --> 01:07:20.500 I'll see you next time 01:07:45.440 --> 01:07:48.100 I think is the norm.