Building the Agentic Software Factory - Thomas Dohmke and Matan Grinberg의 전체 녹취록

day. 00:05:03.040 --> 00:05:04.800 That's not how software ever worked. 00:05:04.959 --> 00:05:24.959 And you know, if if you have um uh worked in in a process like Scrum or Agile or or any waterfall and whatnot, you know that all of that is to some degree made up because we just don't know how long uh it takes to build a certain feature, and it's wildly different between a new codebase, a green field as we call it, or an existing codebase, uh brown field. 00:05:25.120 --> 00:05:28.399 Uh and so these these new factories they need to account for that. 00:05:28.480 --> 00:05:36.800 They need to uh account that even if sometimes you know my cloud code says this will take 15 days to build, and then three minutes later it has built the whole thing, right? 00:05:36.879 --> 00:05:40.399 Because it it's it's using it's using wrong data to estimate how long to take. 00:05:40.560 --> 00:05:48.079 But these new factories need to be designed around that uh as software developers and as businesses, um we we we still want to be profitable. 00:05:48.240 --> 00:05:58.879 We still want to, you know, uh have an OI on the token investment, the energy that we ultimately put into these factories, and I think that's where you know a lot of the innovation will happen over the ne over over the next few months and years. 00:05:59.279 --> 00:05:59.600 Yeah, yeah. 00:05:59.680 --> 00:06:02.800 What's that going to look like from your perspective as you're building this? 00:06:03.199 --> 00:06:19.439 Yeah, I mean, I think the the uh visual that we usually use here when we talk to developers or you know uh engineering leaders about this shift from just agents to software factories is uh we kind of reference, I don't know if you have seen videos of like Tesla's factories. 00:06:20.000 --> 00:06:30.879 But if you look, it's like you know all these robotic arms going and working, and there you don't visually see that many humans in there, but it's still a very complicated engineering problem to assemble that factory. 00:06:31.040 --> 00:06:44.480 And it's just kind of the analog of what we're gonna be doing, but with software, where the human engineers, instead of spending their time writing the code, screwing in the little thing and doing that, it's instead the human engineers are building the factory that builds the software. 00:06:44.720 --> 00:06:54.959 Which is an even more interesting problem, because it's now you're going from let me go and manually do the thing, to how do I abstract away the process that we've been doing for the last 20 years? 00:06:55.120 --> 00:07:01.120 How do I do it in a way such that uh we're not codifying the things that we have done but we shouldn't be doing? 00:07:01.360 --> 00:07:12.480 And how do we instead emphasize the things that we should be doing more of or that you know are lower leverage for humans, and so we want to go and automate that so that then the humans can spend their times time on the things that are way higher leverage. 00:07:12.639 --> 00:07:21.680 And mapping out this software factory and the decision-making criteria that you've built up over time, this is a really, really interesting and frontier engineering problem. 00:07:22.319 --> 00:07:30.639 And most of the people who start you know going towards this future of software factories get to now work on that instead of the kind of boring implementation problems. 00:07:31.360 --> 00:07:34.079 We'll get back to that with the role of the engineer is there. 00:07:34.240 --> 00:07:38.639 But how close are we to having like an autonomous software factory? 00:07:39.839 --> 00:07:41.360 I mean, we have it today. 00:07:41.439 --> 00:07:44.240 It's just how good is it, or how good is the software that's ability. 00:07:44.399 --> 00:07:45.839 It's kind of like software factory. 00:07:46.160 --> 00:07:47.120 How good is factory? 00:07:47.439 --> 00:07:47.600 Sorry? 00:07:47.839 --> 00:07:48.720 How good is factory? 00:07:48.959 --> 00:07:49.120 Yeah. 00:07:49.279 --> 00:07:52.959 Well, I mean, I think with like with factory, you can do this right now. 00:07:53.120 --> 00:07:59.279 And I think software factories are kind of like schedules where everyone has a software factory. 00:07:59.360 --> 00:08:00.160 Everyone has a schedule. 00:08:00.319 --> 00:08:04.800 It just might be very disorganized or very messy or very kind of not formalized. 00:08:05.040 --> 00:08:12.000 So whether you know it or not, even if you were literally writing code on a piece of paper, that is technically a software factory, just a very inefficient one. 00:08:12.399 --> 00:08:21.839 Now, um going and setting up like a you know decades-old bank to be a fully autonomous software factory, that's gonna take a lot of time. 00:08:22.079 --> 00:08:30.319 But if you're building, you know, uh an application from scratch, you can build it in a way such that it is native to this autonomous development. 00:08:30.639 --> 00:08:31.680 You can do that today. 00:08:32.000 --> 00:08:39.360 Um the problem is just as you scale, it's a that much harder because there are different people involved who have their different preferences and different styles. 00:08:39.919 --> 00:08:43.200 How do you decide which ways to standardize? 00:08:43.440 --> 00:08:57.679 Um especially like it, and it's again, it's kind of like GitHub where when you have you know different branches, except instead of branches of code, branches of your software factory, where one person maybe wants a certain process to look one way and the other wants it the other way. 00:08:57.759 --> 00:08:58.720 How do you reconcile that? 00:08:59.120 --> 00:09:08.879 Um so I think there's still some nuances and uh uh interaction patterns, both uh human to human, human to AI, and AI to AI, that we haven't yet nailed. 00:09:08.960 --> 00:09:12.159 Um but we're starting to see it emerge at the frontier, which is pretty exciting. 00:09:12.960 --> 00:09:17.120 Thomas, how close are you to implementing a software

factory at entire? 00:09:18.000 --> 00:09:35.200 I mean building on what Martana said, I think the the fundamental issue is you can you can today run uh agents 24-7 and they can open uh hundreds, if not thousands, of pull requests, both on your internal repos and and your open source, um uh your own open source repos, or even more so on other open source repos. 00:09:35.519 --> 00:09:50.240 And what happens is that those people all will uh uh go crazy about it because you're just increasing the the load that somebody has to review uh uh 10 or 100fold, and um and you're doing that by burning tokens. 00:09:50.320 --> 00:10:03.840 So you could say, you know, in a way you're investing into open source by running tokens, um, agents, harnesses against an open source project, except the maintainer doesn't necessarily want that, uh let alone uh has that as part of their roadmap. 00:10:03.919 --> 00:10:09.120 Uh and now you're forcing them to either spend human capital or token capital to review all this. 00:10:09.200 --> 00:10:12.320 So, one, we haven't actually solved the code review problem. 00:10:12.480 --> 00:10:12.559 Right. 00:10:12.720 --> 00:10:22.639 Um, we're not at a stage, I think, in most organizations, um, uh whether it's the enterprise or uh startups or open source, to just automatically merge or YOLO merge at all. 00:10:22.799 --> 00:10:26.159 Um and um and that's fundamentally a trust problem, right? 00:10:26.320 --> 00:10:35.440 We haven't we have seen enough of these incidents where something got merged and then it broke something in the system of an outage or worse, um uh a data leak. 00:10:35.519 --> 00:10:44.720 Um, and so we're not um and and the more we go into like uh regulated industries and and companies that have uh 30 years of code base, which are more and more companies, right? 00:10:44.799 --> 00:10:54.720 Because now we're heading into the uh uh late 2020s, 30 years ago was the late 1990s, so a lot of the dot-com bubble uh uh companies now have 30 years of code base. 00:10:54.799 --> 00:10:59.279 You can't just run an agent on that, auto-merge it, and and then hope uh uh for the best. 00:10:59.440 --> 00:11:10.159 Um the other one, and Matan alluded to this, is which like um it doesn't make sense to pay five thousand dollars in tokens if I can do it in five seconds myself, right? 00:11:10.320 --> 00:11:19.039 Or it doesn't make sense to have an agent read my emails to then summarize these emails to me and I'm spending time reading the summary instead of just responding uh responding to those emails. 00:11:19.120 --> 00:11:27.600 And there's I think lots of scenarios today where it's kind of cool and magical to use an agent, uh, but at the same time it's it's much more efficient if you if you actually do that yourself. 00:11:27.679 --> 00:11:27.840 Right. 00:11:28.000 --> 00:11:45.759 And finding that balance and figuring out how we can run these factories autonomously over the whole weekend and not waste a lot of tokens and energy and ultimately money, uh, it will be crucial for every business because otherwise I can't run my business if on Monday I come back and I'm proud that it ran all weekend long, but I'm frustrated. 00:11:45.840 --> 00:11:49.919 And you know, I've had moments where I put in my prompt, what the hell did you just do? 00:11:50.080 --> 00:11:51.679 Uh it's oh, I apologize. 00:11:51.759 --> 00:12:00.879 I'm like, I want a refund, and it's like, oh, here's how you open a support request, and all of a sudden the agent can no longer be an agent, and it just sends me to human support at uh at one of the model providers. 00:12:00.960 --> 00:12:07.600 So I think we are there from a technology perspective, but we're not uh actually there from a business perspective. 00:12:07.759 --> 00:12:15.120 And bringing those two uh into alignment, you know, I think that will be the challenge for many of us over the um over the time to come. 00:12:15.519 --> 00:12:15.919 For sure. 00:12:16.000 --> 00:12:19.360 The era of token maxing lasted, what, two months maybe? 00:12:19.440 --> 00:12:20.879 So we're already past that. 00:12:21.120 --> 00:12:26.480 But you brought up code reviews, and you've talked about that in the past too as one of the bottlenecks, right? 00:12:26.639 --> 00:12:27.440 And I think it still is. 00:12:27.519 --> 00:12:30.399 Um, Matan, how do you think about that and how do you handle that? 00:12:30.720 --> 00:12:52.159 Yeah, I mean, I think it's there's kind of this same systematic approach to every part of software development, which is figuring out what are the time sinks that are low leverage and kind of a waste of our engineers' time, and okay, how can we standardize that and automate it as much as possible so they can focus on you know the higher leverage, you know, minutes and hours of their day. 00:12:52.399 --> 00:13:02.480 Now, code review is obviously very high leverage because if you do something wrong, you know, you might uh it might lead to an outage to your customers, you know, very disastrous for the business. 00:13:02.639 --> 00:13:09.120 Um but there's a lot of code review, especially as it was done like five years ago, that is not particularly time efficient. 00:13:09.279 --> 00:13:13.360 Like, you know, oftentimes it takes like hours just to even understand what's actually happening. 00:13:13.519 --> 00:13:20.960 Or there are just certain things where to be methodical and like check every line, like humans aren't best at it, agents are. 00:13:21.200 --> 00:13:42.159 Um and so we uh at like one of our goals is trying to get to a point where as many of our PRs as possible are able to be kind of fully reviewed and auto-merged just with factory on its own, but then also detecting, okay, which are the parts of the code base where we're gonna need a human to uh kind of be in there no matter what. 00:13:42.320 --> 00:13:49.200 Um because I I think it's important to know we don't want to go from just like completely human to just

none at all. 00:13:49.440 --> 00:13:51.600 We just want to have better resource allocation. 00:13:51.759 --> 00:13:59.600 So if we're gonna spend an hour on code reviews in a day, let's just make sure that hour is spent very well and very kind of uh methodical. 00:13:59.840 --> 00:14:07.519 Um but examples, I mean, to to the question earlier about software factories, like the agent that we build is model agnostic. 00:14:08.000 --> 00:14:10.879 These days there's like 10 models that come out every week. 00:14:11.120 --> 00:14:24.960 If we had a human go and make sure you know it works in an optimized fashion for every model that comes out, we would need to hire like a team of a hundred people to just constantly be doing that and then reviewing all of that to make sure it works well, and doing all the evals and then you know putting it into production. 00:14:25.120 --> 00:14:29.039 That is something that we've built a fully autonomous software factory loop for. 00:14:29.200 --> 00:14:33.039 Where we used to have engineers working on that, now we don't, and it can even auto-merge. 00:14:33.200 --> 00:14:40.240 So a new model comes out, it'll go through our the software factory, it'll go through the evals, it'll iterate based on it. 00:14:40.399 --> 00:14:44.559 If it sees certain ones aren't performing well, it'll fix those, merge it into production. 00:14:44.799 --> 00:14:52.080 Then there's certain things of like, you know, core part if there are changes to the core agent loop, we're not gonna auto-merge those. 00:14:52.240 --> 00:15:07.039 We're wanna we're gonna wanna get whichever engineer is responsible for that, kind of as much important context as possible and as much focus as possible, so they don't need to worry about the thousand other lines of code that don't really matter to the crux of the change that's happening. 00:15:07.200 --> 00:15:10.720 Um yeah, we we we really think about it as a resource allocation problem. 00:15:10.879 --> 00:15:18.559 If we're gonna spend a certain number of minutes on this review, what where should that human spend their time and how do we get them up to speed as quickly as possible? 00:15:18.960 --> 00:15:20.799 Is that what you do at Entire as well? 00:15:20.960 --> 00:15:23.759 How much have you automated there in the review process? 00:15:24.320 --> 00:15:30.639 It's not automated uh in the way that an agent can automatically merge with no human supervision. 00:15:30.720 --> 00:15:30.879 Yeah. 00:15:31.200 --> 00:15:36.559 Uh but it is automated in the sense that almost nobody reads every single line of code anymore. 00:15:36.720 --> 00:15:47.600 To give you a real example from this morning, um uh in the last few weeks, a friend of ours took our company uh robot, uh, we call him Marvin and made like a real 3D printed robot with electronics and all that. 00:15:47.679 --> 00:15:54.399 And the all the code is written uh uh with codecs, and um uh uh you know you built a GitHub repo uh and we made that. 00:15:54.480 --> 00:16:01.039 I wanted to make this public this morning so when later he demos it on the stage uh on the booth, uh he can say just go to the repo. 00:16:01.200 --> 00:16:10.960 And when I was clicking the change visibility button on GitHub, uh I was thinking, hmm, maybe I should check that there are no open AI API tokens in the code base, right? 00:16:11.120 --> 00:16:15.120 And I was literally about to leave my hotel to walk over for the session. 00:16:15.440 --> 00:16:28.639 And sure, you know, I I could have postponed it and read every single line of code, or I could have tried to figure out how to install a secret scanner on my computer, or and that's what I did on my phone with codex remote control. 00:16:28.879 --> 00:16:45.440 I just said, you know, scan this repo for any uh uh secret leaks and similar things, and it uh ran through, I think, seven secret scanners and it scanned the code base and also figured out there's some minor issues that have nothing to do with the secrets leak, but it gave me by the time I arrived here the the green light, there are no secrets in the code base. 00:16:45.679 --> 00:16:49.679 Uh and then I clicked the button, and so uh Stefano can can demo it later. 00:16:49.919 --> 00:16:52.480 And that's automation, right? 00:16:52.559 --> 00:16:59.759 Like I did uh I offloaded a process that honestly I don't want to do, you know, so I have more time for the things you know that I do want to do, like like this session. 00:17:00.159 --> 00:17:02.399 But how many tokens did you burn just to do that? 00:17:02.720 --> 00:17:05.039 I mean, I it was you know within the credit limits I have. 00:17:05.599 --> 00:17:16.640 That's uh because the agents are now smart enough to uh not try to read every single line of code um uh and understand is it a secret or not, they're just installing installing a secret scanner as well. 00:17:16.799 --> 00:17:21.119 In fact, the whole firmware uh for the robot was all written uh by Codex. 00:17:22.000 --> 00:17:29.279 And and I installed the code base on Monday on my computer so I can flash it myself and install all the tools to flash the microprocessor and all these things. 00:17:29.440 --> 00:17:34.079 But the point I want to make is that is there still a need for developers to understand code? 00:17:34.240 --> 00:17:41.599 Absolutely, because you need to find the moments when the agent does stuff that that is bad for your business or bad for the customer. 00:17:41.759 --> 00:17:50.960 Uh but are we uh better off when AI reviews every single line of code uh in a sh ever-increasing code base over us reading every single line of code? 00:17:51.119 --> 00:17:51.599 Absolutely. 00:17:51.680 --> 00:17:59.039 Like there's no way, you know, on a Friday night uh before you want to head in the weekend that you're finding exactly all the issues that the agent can find. 00:17:59.200 --> 00:18:04.319 In fact, your human nature will mean you're finding three or four or five, and then

you feel good about it. 00:18:04.400 --> 00:18:09.200 You had enough dopamine, and you can kind of say, look, you know, I found something, so clearly I did it. 00:18:09.359 --> 00:18:14.640 Um, and uh and now I can hand off the work to or back to my my coworker who who authored the pull request. 00:18:14.960 --> 00:18:15.680 Yeah, yeah. 00:18:16.000 --> 00:18:25.119 Now, one thing, Mitan, you kind of hinted at it is what how does the role of the developer change in all of this, right? 00:18:25.200 --> 00:18:33.039 Because this morning I was talking to a bunch of people here, and I it was pretty easy to send some anxiety among developers, right? 00:18:33.279 --> 00:18:34.000 What's next? 00:18:34.160 --> 00:18:39.039 How does this influence their role going forward, or does their role even exist going forward? 00:18:39.200 --> 00:18:42.799 So you kind of hinted at how you're thinking about that, but maybe go a little bit deeper there. 00:18:43.200 --> 00:19:07.359 Yeah, I mean, I think first of all, what got most people into engineering or software development in the first place, and what makes the best engineers and software developers is people that like solving problems, that like systems thinking, thinking around multidimensional constraints where the solution is really not obvious, and you have a lot of different trade-offs and a lot of kind of different constraints to be considering. 00:19:07.599 --> 00:19:20.000 Those are like the interesting problems that people are excited about in the first place, or owning end-to-end outcomes, like building some new kind of entirely new product, or delivering some new experience to users. 00:19:20.240 --> 00:19:23.039 That's been that's what initially got people into engineering. 00:19:23.519 --> 00:19:30.880 But then the reality of engineering for the last 20 years, especially in large companies, is you own like this little thing. 00:19:31.039 --> 00:19:40.160 We can only ship three features in this quarter, and you must own this little thing in this part, like you know, you're you're a cog in the machine or you do this little thing and that's it. 00:19:40.240 --> 00:19:54.640 Um and then it turns out that in order to go and solve these constraints or go and you know implement these solutions to these problems, you need to know the secret incantations to whisper to the computer in order for the computer to do the thing that you want it to do. 00:19:54.799 --> 00:20:00.799 Um now, in this new world, uh it turns out that you don't need to whisper the secret incantations anymore. 00:20:00.960 --> 00:20:13.759 Now, a lot of us feel very sentimental about the incantations that we've spent decades learning and understanding and you know, growing fond of, but we kind of need to remember what initially got us into this discipline in the first place? 00:20:13.920 --> 00:20:23.759 Was it we really like the sound of the incantations, or we like solving the problems and delivering the outcomes to the users or to our friends or whoever we're building for? 00:20:23.920 --> 00:20:30.480 Um, and so I think there is a little, you know, it is always jarring to like be used to something for such a long period of time and now it's changing. 00:20:30.559 --> 00:20:40.720 But I think the biggest thing is just like remembering, like, and I actually ask this of every single engineer I interview, and even users, is I always like to hear like, why did you get into engineering in the first place? 00:20:40.880 --> 00:20:51.680 And it's always something like, oh, well, when I used you know the Atari back in you know the 90s, or the I was playing these video games, and I love it's always things that are way bigger, not like, man, I really loved the syntax of C. 00:20:52.319 --> 00:20:55.039 And that just like like no one ever says that. 00:20:55.279 --> 00:21:00.400 Um, and so I I think we you know we kind of need to refocus on like why did we do this in the first place? 00:21:00.480 --> 00:21:06.480 And now these tools will allow us to go and do that in a way higher leverage manner, which I think is really exciting. 00:21:06.640 --> 00:21:09.039 Like, we will solve so many more problems. 00:21:09.279 --> 00:21:18.960 I think people who say, like, oh, you know, engineers are going away, inherently or implicit in that belief is that basically there is a finite number of problems and we're solving all of them. 00:21:19.039 --> 00:21:20.720 So now there's no more need for engineers. 00:21:20.880 --> 00:21:24.799 I would argue there are a lot of problems in the world and we're solving a very small amount of those. 00:21:25.119 --> 00:21:37.200 And I think the good news is people like engineers who are the best problem solvers in the world will now go and solve larger, more ambitious problems and get them done faster, and I think that's a great world to be living in. 00:21:37.680 --> 00:21:45.599 Just to push back a little bit there, I think it a lot of people just love the craft of building code, right? 00:21:45.759 --> 00:21:45.920 Yeah. 00:21:46.799 --> 00:21:50.720 They might not want to become architects or something else now. 00:21:50.799 --> 00:21:52.160 What do you tell them? 00:21:53.920 --> 00:21:57.680 I tell them to keep loving the craft and evolving the craft. 00:21:57.759 --> 00:22:06.640 And just because you know I have now an Agent that can write code for me doesn't mean I cannot find time during my day to evolve my craft. 00:22:07.039 --> 00:22:10.720 In fact, I've been talking on these kind of stages a lot about that. 00:22:10.799 --> 00:22:14.720 We need to teach all children coding early on. 00:22:14.799 --> 00:22:25.519 Like we can argue is it a first grade or third grade thing, but like in the same way, like why would I not learn coding, but I learn about Maxwell equations and Newton's laws and how a cell is constructed. 00:22:26.079 --> 00:22:36.160 In many ways, you know, coding is now an essential part of our life because we're all consumers of

software on our devices that we carry with us, you know, uh at 24-7. 00:22:36.400 --> 00:22:38.559 When if you have a watch that you know tracks your sleep. 00:22:38.720 --> 00:22:43.200 So coding is something that we should all learn and evolve our crafts. 00:22:43.279 --> 00:22:54.640 And there's people that are really, really good at this in the same way that people, there's people out there, and you can watch them on YouTube that are really good at manual woodworking, uh, and obviously manual making music, acoustic music, and and all that. 00:22:54.880 --> 00:22:56.559 And um, and so that I think that's number one. 00:22:56.880 --> 00:23:00.640 I don't think the craft goes away just because we have a machine. 00:23:00.720 --> 00:23:04.319 Uh, and the same for what it's worth with many other things on the planet, right? 00:23:04.400 --> 00:23:09.759 You uh like to cook yourself even though you can get a pre-made meal uh in the supermarket. 00:23:09.920 --> 00:23:19.599 Um I think the second part uh is that um look, you know, the uh we as developers we've always evolved and we've always found ways to build more. 00:23:19.759 --> 00:23:22.880 Because let's be honest, my backlog is longer than ever. 00:23:23.039 --> 00:23:27.039 And I don't know any company who's coming to me and saying, Oh, we have no more ideas. 00:23:27.119 --> 00:23:29.119 You know, we have we ran out of items in our backlog. 00:23:29.200 --> 00:23:32.880 We have no more bugs, no security vulnerabilities, no feature requests, no, no. 00:23:33.440 --> 00:23:36.400 The only people that have that is those that have no product. 00:23:36.559 --> 00:23:38.960 And everybody else has an ever-growing backlog. 00:23:39.200 --> 00:23:45.680 And I don't think you know that we have found the factory that can you know get through the backlog faster than we can uh generate ideas. 00:23:45.759 --> 00:23:49.279 And and so every software developer will be a developer. 00:23:49.359 --> 00:23:52.799 That means they take ideas and convert them into software. 00:23:52.960 --> 00:24:02.480 And they will be able to code and they will be able to understand code at a certain level, you know, who who still you know in the room uh understands COBOL or assembly language and those kind of things. 00:24:02.640 --> 00:24:08.079 But um, they will every day um you know take more and more ideas uh uh and and make real products out of them. 00:24:08.160 --> 00:24:13.279 And and for me at least that was always the dream ever since I saw a computer in the in the late 1980s. 00:24:13.759 --> 00:24:17.599 Now you've talked about the idea of the middle class engineer, right? 00:24:17.759 --> 00:24:21.440 Talk about that a little bit more and what happens to those. 00:24:22.559 --> 00:24:33.839 Every engineer you know that has been in this industry uh uh has uh con has seen that there is um you're you're already you're all you're only preserving your job by constantly learning. 00:24:34.000 --> 00:24:44.559 Um we have gone through these uh different phases, you know, I already mentioned assembly language and came basic and I learned Pascal, uh, but it was before the internet, so you did it all learned by yourself and my first company. 00:24:44.640 --> 00:24:48.960 We shipped floppy disks uh in in envelopes uh to people uh in Germany. 00:24:49.119 --> 00:24:57.359 And then came the internet, and all of a sudden, not only could you uh chat with others and ask them questions in the middle of the night, uh you could also download open source, right? 00:24:57.440 --> 00:25:03.680 That's that's really the the big shift that we went through in the late 1990s and then with GitHub again in the in the 2010s. 00:25:03.839 --> 00:25:08.000 Then obviously came the cloud and mobile devices, and so we have always this before and after. 00:25:08.079 --> 00:25:17.440 And and today, when you when you're running a startup, you don't have to buy servers, you don't have to think about data center space, network cables, you're not you're not hiring people to manage all of that. 00:25:17.519 --> 00:25:23.440 Uh you can just spin up you know uh a Kubernetes cluster on any of the cloud providers um and and get up and running. 00:25:23.519 --> 00:25:27.839 And I think those transformations we have gone through many, many times, and this is just one of them. 00:25:27.920 --> 00:25:32.880 It's a big one because it touches the thing that is dear and near to many of our hearts, uh code. 00:25:32.960 --> 00:25:39.119 Uh but I think it also uh has brought so much fun and so much excitement to the to the scene. 00:25:39.200 --> 00:25:56.400 And if you look at uh companies like Lovable, it has also brought you know hundreds of well not millions, hundreds of thousands um uh new people into the craft that will, because they have seen how easy it is and how fun it is to build software, naturally also start learning a little bit more about the code that Lovable generates. 00:26:01.519 --> 00:26:11.440 He has seen them in the computer museum and how many junior engine engineers have you hired in the last year? 00:26:12.640 --> 00:26:24.640 I think the it's funny, the the differentiated value of someone who has decades of experience is actually less now than it was before. 00:26:25.200 --> 00:26:36.400 Um because you know, right now I think it is more accessible than ever for someone who is just very high agency to go and solve problems. 00:26:36.559 --> 00:26:49.920 I think traditionally, and people talk a lot about the roles of product and engineering and merging and whatever, I think generally the best product people and the best engineers are good at solving problems that have multi-dimensional constraints. 00:26:50.079 --> 00:26:54.240 The only difference is the engineers know the secret incantations and the product people don't. 00:26:54.400 --> 00:26:55.680 That's basically the only difference. 00:26:55.839 --> 00:26:57.839 Otherwise, they have a lot of common traits. 00:26:58.000 -->

00:27:16.160 And I think similarly, uh the best builders these days, whatever label you want to put, you know, engineer, software, developer, whatever, the things that matter most are just high agency and just like IQ, raw, firepower, whatever you want to describe it, just like you know, they're smart and they have high agency. 00:27:16.319 --> 00:27:20.160 Um you can be 30 years into your career and satisfy those constraints. 00:27:20.400 --> 00:27:23.680 You can be two months into your career and satisfy those constraints. 00:27:23.839 --> 00:27:35.920 I think that the biggest thing is how do you identify the one who is earlier in career and high agency, and how do you identify the one who's you know 30 years into their career and low agency or high agency? 00:27:36.079 --> 00:27:40.400 Because sometimes when you have a lot of experience, you can mistake that experience for agency. 00:27:40.640 --> 00:27:42.480 Um and I think that's the biggest thing. 00:27:42.640 --> 00:27:51.359 But uh years of experience matters so much less now, generally, than unless you're doing some hyper-specific use cases than ever before. 00:27:51.680 --> 00:27:52.000 Right. 00:27:52.720 --> 00:28:00.559 Final question, like for the audience members here, what should they do to prepare for this new world of software factories? 00:28:01.599 --> 00:28:11.200 We already uh checked the first box, which is attend the conference and uh learn about all the uh uh uh new developments in technology and all the companies uh on the show floor. 00:28:11.359 --> 00:28:19.519 But I think it's um just use AI and try to build something uh with an agent um uh every single day. 00:28:19.680 --> 00:28:23.200 And I think a lot of um uh uh Martan mentioned agency. 00:28:23.279 --> 00:28:26.319 I think autonomy uh is the other piece that plays into that. 00:28:26.480 --> 00:28:37.759 You need to learn how you actually prompt it and plan uh the agent loop so it can run through the session uh uh or through the through your day or hopefully through your night. 00:28:37.920 --> 00:28:45.599 It's not an achievement if we wake up every 30 seconds or every every every every three hours uh in the night or during a meeting to give it another prompt. 00:28:45.759 --> 00:28:47.759 That's not how we want to uh design these loops. 00:28:47.839 --> 00:28:56.559 You want to be able to design of them, and I think um uh the ones that are leading uh the industry right now are the ones that have practiced this a lot over the last year or so. 00:28:56.720 --> 00:29:00.559 And uh we ourselves need to also ramp up and and and and and try it out. 00:29:00.640 --> 00:29:02.880 And that's that's very similar to how we learn coding. 00:29:02.960 --> 00:29:08.880 You become a good senior or principal developer by learning and practicing the craft of coding. 00:29:09.039 --> 00:29:13.680 Now we have to learn and practice the craft of managing uh fleets of agents. 00:29:14.319 --> 00:29:31.680 Yeah, and I think just seeing you know, friends, you know, people who have joined factory, you know, colleagues, all of this, uh the people who end up doing the best and kind of scaling in this new era are the people that embrace, genuinely embrace the uncertainty and the high variance of everything that's happening right now. 00:29:31.839 --> 00:29:35.039 And instead of being uncomfortable by it, just diving into it. 00:29:35.200 --> 00:29:41.440 It's like I have no, like it's okay to embrace and say, I don't know what exact skill is gonna be most important. 00:29:41.519 --> 00:29:44.960 I'm just gonna try doing it this way, try doing it that way, see what I like. 00:29:45.119 --> 00:29:57.599 Instead of like every generation ever, every single generation for the last thousand years is always like, oh, this next generation, they're messing it up, they don't know what they're doing, they're losing the important things that only we understand now. 00:29:57.839 --> 00:30:05.519 Um and just remember that, and even despite that, you might find that you're right, but just go and experiment and see what is this kind of new era. 00:30:05.680 --> 00:30:07.359 Is it real or is it BS? 00:30:07.680 --> 00:30:12.640 Um, and by doing that, you're gonna stay at the frontier and learn and then you know be able to teach others. 00:30:12.720 --> 00:30:17.279 And I think just embracing that discomfort and uncertainty is probably the the most important thing. 00:30:17.680 --> 00:30:18.240 All right. 00:30:18.480 --> 00:30:20.880 We shall go ahead and embrace the uncertainty. 00:30:21.119 --> 00:30:21.680 Thank you both. 00:30:21.920 --> 00:30:22.640 Back to the agents. 00:30:23.519 --> 00:30:24.240 Thank you.