Full Transcript for The Platform Engineers' Cookbook

and they are essentially there to enable developers within an organization to self-service themselves. 00:03:36.039 --> 00:03:58.212 So if platform engineering, rather than having DevOps teams perform ticket ops and take care of making user developer requests being output, the platform engineers essentially provide them with a sort of self-service capability so they can request what they want through some sort of portal or interface. 00:03:58.212 --> 00:04:05.151 Yeah, so that's pretty much what platform engineering is, and platform engineers are another product team within the organization. 00:04:05.151 --> 00:04:26.757 However, the core distinction is, rather than building a product where, in a traditional software sense, the product is the software that you are providing to end customers of your organization, platform engineers deliver a product where the end users of the product are the developers within the organization of the product are the developers within the organization. 00:04:26.757 --> 00:04:33.266 So our customers are the developers and that is where we get our requirements from what we have to build and where we get feedback on how to improve our products. 00:04:33.867 --> 00:04:40.346 Yeah, I noticed that both of you in your introduction you mentioned Kubernetes. 00:04:40.346 --> 00:04:44.029 Where is that coming in and what does that entail? 00:04:47.160 --> 00:04:50.968 So Nicholas has a very, very nice definition of of kubernetes. 00:04:51.088 --> 00:05:02.413 Um, yeah, that he needs to share yeah, so I think, a definition of kubernetes I like to coin or talk about a lot and which I really like to explain. 00:05:02.413 --> 00:05:04.456 Kubernetes, as is traditionally. 00:05:04.456 --> 00:05:43.920 When a lot of people who are have an idea of what Kubernetes is, they like to think of Kubernetes as something that runs my containers or orchestrates my containers, but it's really more than that and I think a term that sums up Kubernetes really well in this current state of where it's at, in its, you know, overall software life cycle is a universal and extensible cloud-native API that happens to come with a container orchestrator baked in, and I think that's really crucial because that allows us to think of Kubernetes not as something that runs my container, but really a universally acceptable API standard that everybody can agree upon. 00:05:43.920 --> 00:05:55.127 And, as a matter of fact, the Kubernetes API is, to date, the only universal API in the whole cloud-native ecosystem that everybody can agree on. 00:05:55.480 --> 00:05:58.127 Yeah, and where does Crossplane come into play then? 00:05:59.550 --> 00:06:00.273 André, do you mind? 00:06:01.740 --> 00:06:14.331 Yeah, so Crossplane comes into play based on the fact that, as Nicholas said, Kubernetes is a hyper-extensible API with a standard, so what cross-planning allows us is to utilize those things. 00:06:14.331 --> 00:06:28.788 So cross-planning allows us to define resources in the AWS world which might be an instance of database and then make it conform to a standard set of API rules when an end user needs to consume them. 00:06:29.028 --> 00:06:32.206 There's a standard set of rules and he just needs to follow those rules. 00:06:32.206 --> 00:06:40.886 He does not have to be aware of the specific API layout of AWS or AKS or even of Memstor. 00:06:40.886 --> 00:06:43.891 He just needs to be aware of the Kubernetes API. 00:06:43.891 --> 00:06:55.343 It makes it extremely flexible and also easy to transport that knowledge across multiple teams, and that is where Crossplane shines, in a large part because of Kubernetes. 00:06:56.906 --> 00:07:00.870 Yes, and also from that with Andrew just described. 00:07:00.870 --> 00:07:07.920 It's really from a developer perspective, the fact that the API is one API that they have to communicate with. 00:07:07.920 --> 00:07:22.644 Obviously, that takes care of abstracting well of the complexities of these high-fidelity APIs offered by, for example, aws or Azure Azure. 00:07:22.644 --> 00:07:29.326 What a platform engineer is then responsible for doing with Crossplane as their platform orchestrator tool is to develop Crossplane compositions, which are essentially how they tell Crossplane. 00:07:29.326 --> 00:07:34.637 It's essentially how they configure the logic of the API. 00:07:34.637 --> 00:07:49.136 So once an instance of an API is created or updated, how does Crossplane take that form of the user's request and translate that into actual, real-world resources in external vendors like, for example, aws? 00:07:50.168 --> 00:07:57.312 Is my reading correct that it almost sounds that there's a lot of empowerment to the developer, almost like a self-service? 00:07:57.312 --> 00:08:02.454 If this is the case, I'm wondering are you not giving too much freedom? 00:08:02.454 --> 00:08:09.853 Because how does that actually compensate against security governance rules that you apply as a company? 00:08:10.564 --> 00:08:16.973 So that's one of the most fantastic things about Crossplane, if you look at Crossplane's lifecycle. 00:08:16.973 --> 00:08:21.213 First of all, we do well-architected golden paths. 00:08:21.213 --> 00:08:31.536 If a developer needs a database, for example, we've already architected a database setup that is compliant with the company's security and risk profile. 00:08:31.536 --> 00:08:34.211 A user cannot step out of that. 00:08:34.211 --> 00:08:36.566 That's one of the main things. 00:08:36.566 --> 00:08:42.375 And, as Nicholas also will describe now, the end user not. 00:08:42.375 --> 00:08:48.097 There's no information leakage, so the user does not receive any information that is not pertinent to him. 00:08:49.470 --> 00:09:07.956 Nicholas can describe the old terraform way of of why you also see a lot of information yeah, so traditionally with isc tools like terraform, like Anjo just said, there is domain-level knowledge that gets leaked onto developers

when you, for example, create an instance of a Terraform module. 00:09:08.404 --> 00:09:18.096 What I refer to that is that developers are still, in theory, directly interfacing with the hyperscalers as APIs directly. 00:09:18.096 --> 00:09:28.735 Like we mentioned before, these APIs are high fidelity but they give people the freedom to do pretty much whatever they want, so there's not really any guardrails put on top of those APIs. 00:09:28.735 --> 00:09:30.211 With Crossplane. 00:09:30.211 --> 00:09:34.094 What we want is we don't want developers to add all interface with these APIs. 00:09:34.094 --> 00:09:55.052 What we want is we want Crossplane to be the middleman in front of these cloud vendors and, like André said, these golden paths that have the security and risk profiles baked in to make sure that these building blocks we're building with Crossplane using compositions are compliant within the organization and that they are production ready. 00:09:55.052 --> 00:10:01.436 That way, developers don't need to go and figure out how to make their infrastructure production ready and compliant. 00:10:01.436 --> 00:10:06.113 They only create an instance of these custom APIs that are exposed over the Kubernetes API. 00:10:07.446 --> 00:10:15.794 Would you mind to, because there's a lot of jargon in here, and one of the things that stood out for me was that term compositions in cross-plane. 00:10:15.794 --> 00:10:16.950 What does that actually mean? 00:10:17.904 --> 00:10:20.363 So I think we have to step back a wee bit. 00:10:20.363 --> 00:10:24.269 Right, we're Going back to platform and engineering and use analogy for it. 00:10:24.269 --> 00:10:41.077 So if we think of platform engineering as a restaurant, you as the end client don't want to come in and then select your salad, your bread, your sauces, actually have to go to the farmer, get the milk and the bread and all of that. 00:10:41.077 --> 00:10:46.496 You, as an end client, want to actually walk into a restaurant and get a curated menu for you. 00:10:46.496 --> 00:10:53.759 That is the role of platform engineering essentially is creating these curated menus for you to be easily consumed. 00:10:53.759 --> 00:11:00.768 So now we're going to think about compositions At a base level. 00:11:01.490 --> 00:11:03.316 In CrossPen you get things called providers. 00:11:03.316 --> 00:11:07.255 Providers you can see as the ingredients for my recipe. 00:11:07.255 --> 00:11:13.817 So if we're making a salad, it might be the salad itself, the sauce on the salad, the tomatoes, the cheese. 00:11:13.817 --> 00:11:16.150 They are the fundamental building blocks. 00:11:16.150 --> 00:11:17.535 Then we take a composition. 00:11:17.535 --> 00:11:19.506 Composition is basically our recipe. 00:11:19.506 --> 00:11:27.673 It tells us we're going to need so much cheese and we're going to need it in this order and that step to get our final product. 00:11:27.673 --> 00:11:30.255 That is a good analogy for what a composition actually does. 00:11:30.255 --> 00:11:36.020 It's a recipe with our base ingredients to get our final product delivered to our client and in your example, the Caesar salad would be the product. 00:11:36.041 --> 00:11:38.583 Yes, yeah, in your example, the Caesar salad would be the product. 00:11:38.583 --> 00:11:39.403 Yes. 00:11:42.729 --> 00:12:02.172 Yeah, so if we consider the API the API in Crossplane's case, that is, I suppose, of the Kubernetes API we could refer to that as the waiter, the person taking the request Under the hood, the stuff happening in the kitchen, that's the stuff that the platform engineers takes care of configuring, building up the recipes in the kitchen. 00:12:02.172 --> 00:12:14.655 That's the stuff that the platform engineers takes care of configuring, Building up the recipes in the form of compositions and configuring cross-plane with the providers, the ingredients, like Andre said, to orchestrate the stuff in the external providers like, for example, aws. 00:12:14.655 --> 00:12:23.831 So yeah, from a developer perspective, the developer is the person coming into the restaurant, looking at the menu, saying they want X, y, z. 00:12:23.831 --> 00:12:32.393 They make the request to the waiter in this case the API under the hood, the composition and the configurations that the platform engineers have done. 00:12:32.393 --> 00:12:39.653 In other words, the kitchen staff take care of preparing that and, lo and behold, out comes your requested items. 00:12:41.035 --> 00:12:41.798 Super interesting. 00:12:41.798 --> 00:12:50.264 I was looking at Hal's kitchen from Gordon Ramsay and obviously in every kitchen and every analogy there's like pitfalls. 00:12:50.264 --> 00:12:55.857 So what would be the most common pitfalls in a setup that you just explained? 00:12:57.645 --> 00:13:14.081 So I think first thing that I want to mention when it comes to Crossplane is that an organization wanting to adopt it has to consider the initial investment and skill required and able to maintain and operate Crossplane effectively. 00:13:14.081 --> 00:13:23.004 As we know, crossplane is built on top of Kubernetes and Kubernetes by itself is a beast, is built on top of Kubernetes and Kubernetes by itself is a beast. 00:13:23.004 --> 00:13:30.894 It does require a lot of experience and skill to run a Kubernetes cluster in production at scale, but also Crossplane as an additional learning curve. 00:13:30.894 --> 00:13:47.259 So for me, I would say larger organizations with hundreds or even thousands of engineers would benefit from having a platform backed by this orchestrator, in which case we're referring to crossplane just because it's a cloud native tool, um. 00:13:47.259 --> 00:14:03.312 But if you are an organization of like 10 engineers, um, the investment of adopting crossplane would not be worthwhile because the value add and putting together a team to maintain that isn't going to be justifiable given the size of the

organization. 00:14:03.605 --> 00:14:18.330 So that's number one, and I then also think care needs to be taken to make sure that the people maintaining it are well skilled and are trained to be able to run with this sort of setup in a production environment. 00:14:20.510 --> 00:14:36.846 Now using your Hell's Kitchen analysis, one of the pitfalls you see in Hell's Kitchen is generally the lack of standards and the lack of skills in the staffing, which is where Gordon Ramsay comes in and tries to fix everything, standardize it and all that. 00:14:36.846 --> 00:14:47.423 And that's the same with Crossplane, since Crossplane is immensely malleable it is meant to fit everyone's needs. 00:14:47.423 --> 00:14:53.182 You sometimes fall into the trap that basically everyone goes off and does their own thing. 00:14:53.182 --> 00:14:59.043 So the chef or someone thinks he's making a Caesar salad, but meantime it's actually a lasagna. 00:15:00.591 --> 00:15:18.705 Big difference though yeah, so one of the big difference though so one of the key concepts that makes one of the pitfalls, but also to to fix that is, to start standardizing on your implementation of crossplane is deciding on a core, core key concepts that you can apply on crossplane company one. 00:15:18.705 --> 00:15:24.779 This makes that knowledge sharing as well could becomes a lot easier, since everything is standardized. 00:15:26.023 --> 00:15:33.020 Yeah, I also want to add on that analogy quickly and I want to refer to fast food chains like McDonald's. 00:15:33.020 --> 00:15:42.496 Let's quickly consider and I heard this the other day when watching a video on YouTube, but a guy was basically saying why is McDonald's so successful? 00:15:42.496 --> 00:15:52.461 Why do people eat McDonald's if it's not as great as, maybe, making a home-cooked meal themselves or going to a five-star restaurant? 00:15:52.461 --> 00:15:53.735 Well, number one it's cheap. 00:15:53.735 --> 00:15:56.038 Number two, it's fast. 00:15:56.038 --> 00:15:58.697 You get your meal fast, but also it's consistent. 00:15:58.697 --> 00:15:59.500 That's important. 00:15:59.500 --> 00:16:01.330 You get a consistent experience. 00:16:01.330 --> 00:16:10.556 You can go to McDonald's at X location or Y location, but you'll get the same food and you know what you've had is going to meet these sort of statins. 00:16:10.556 --> 00:16:13.220 You're happy with that. 00:16:14.221 --> 00:16:15.042 Predictability. 00:16:15.964 --> 00:16:17.365 Predictability exactly. 00:16:17.365 --> 00:16:18.696 You know what you're going to get. 00:16:18.696 --> 00:16:20.152 So that's why it's so popular. 00:16:20.152 --> 00:16:23.716 So that's why it's so popular If we consider how McDonald's does things. 00:16:23.716 --> 00:16:40.039 If you've ever watched the movie the Founder, there's the scene where he's basically they're on a tennis court and they're laying out how they want to structure the restaurant to make sure that the time in which they can

pump out you know, food for their customers is as fast as possible. 00:16:40.039 --> 00:16:43.802 You know the efficiency of the overall kitchen design is as smooth as possible. 00:16:43.802 --> 00:16:47.635 You know the efficiency of the overall kitchen design is as smooth as possible, such that they can serve their customers at a much faster pace. 00:16:47.635 --> 00:16:53.977 That is, in a sense, if we had to take into account what platform engineering is trying to do. 00:16:53.977 --> 00:16:55.154 That's what we're trying to do. 00:16:55.154 --> 00:17:14.921 We're trying to make our developers get their infrastructure and related services created at a record speed, and that was, if they have this, then the question is yes, they can go to AWS directly and create this stuff, but now they have this McDonald's interface, if I were to call it that. 00:17:15.289 --> 00:17:20.102 That's consistent, that's reliable and they know it works and that's cheap. 00:17:20.102 --> 00:17:46.457 So they would a lot of people would start using that and, yeah, I think it draws a lot of inspiration from one like the fast food industry, but also even just like the automobile car production industry, Like, for example, how Henry Ford revolutionized the automobile industry with his way of constructing cars at a record pace and selling cars or producing a large amount of cars in a small amount of time. 00:17:47.932 --> 00:17:56.717 Guys, we're almost at the end of this podcast recording and I learned a lot today and I'm wondering if each of you could give like a one tip to the audience. 00:17:56.717 --> 00:17:57.480 What would that be? 00:17:57.849 --> 00:18:00.660 I would say standardization. 00:18:00.660 --> 00:18:04.542 It is probably your greatest ally of all times. 00:18:04.542 --> 00:18:08.420 A good example for that would be actually the Industrial Revolution itself. 00:18:08.420 --> 00:18:15.731 So one of the greatest things that came out of the Industrial Revolution was the metric and imperial systems for standardization and that created. 00:18:15.731 --> 00:18:21.714 So if you had a bolt that was broken, you did not have to custom make it, you could just go to the shop and buy it. 00:18:21.714 --> 00:18:25.702 And that in general is the best advice for platform engineering. 00:18:25.702 --> 00:18:32.499 It is creating standardization throughout the organization for easily transferring knowledge between people. 00:18:33.320 --> 00:18:54.506 Yeah, yeah, For me I think, like André just said, to add on that, I think it's important for a guy considering a career as a platform engineer to realize that he is building a product and he needs to make his product as easily usable as possible for their customers, which is developers. 00:18:55.230 --> 00:19:04.164 So he has to think of how can a developer use my platform to do his job and get his things requested in the easiest way possible? 00:19:04.164 --> 00:19:24.080 Now, another thing I want to add is a common pitfall we see with a lot of our clients that are trying to start building internal developer platforms or portals is that they are not trying to follow what is the community has now pretty much coined as the so-called API first approach platform. 00:19:24.080 --> 00:19:40.599 They are really relying on current trends and tools of basically having IAC tools like Terraform with pipelines fronted by some form of portal, maybe backstage, and really this starts to break down again at scale. 00:19:40.599 --> 00:19:50.175 Having this API in front of all your external platforms is really the key for building a scalable and secure internal developer platform. 00:19:50.175 --> 00:19:56.778 So I think anybody that wants to build a platform with an organization always go API first. 00:19:56.778 --> 00:20:00.336 That's one thing I want to make sure people are aware of. 00:20:01.471 --> 00:20:07.760 Gentlemen, thank you very much for your contribution and all these nice words about platform engineering. 00:20:07.760 --> 00:20:09.556 Thank you very much, everyone. 00:20:09.556 --> 00:20:13.199 Stay tuned for more updates in BBD Tech Unboxed. 00:20:13.640 --> 00:20:14.501 Thank you very much, everyone. 00:20:14.990 --> 00:20:15.810 Okay, thanks, thank you.