文字稿 🔗
WEBVTT
00:00:00.320 --> 00:00:11.359
Now that it's agents doing the work and tokens and the cost of those, everyone's gonna be really like keen on making sure the software development lifecycles was as effective and as efficient as it possibly could be.
00:00:11.519 --> 00:00:15.919
But when it was humans, it was like, oh well, Timmy's just lazy and that's what that's why stuff isn't getting delivered.
00:00:16.000 --> 00:00:20.800
And they didn't focus enough on process and you know value stream mapping and all those important things.
00:00:21.039 --> 00:00:25.600
Now it's when it's agents, it's like, well, that's gonna cost us money, and we can easily see that it's costing us money.
00:00:26.000 --> 00:00:30.079
Most people know me from the DevOps transformation and the cloud native transformation.
00:00:30.320 --> 00:00:36.399
And I love every new chaotic period in the industry because that's where the learning is happening.
00:00:36.479 --> 00:00:38.640
And right now that is AI encoding.
00:00:38.880 --> 00:00:44.079
The way I think about it is I go back to pre-gen AI, and I imagine I have all the manpower and all the time in the world.
00:00:44.159 --> 00:00:46.960
And basically with that mindset, we say, okay, now you have Gen AI.
00:00:47.280 --> 00:00:54.000
DevAcient, there is no excuse not to do everything you ever thought you'd want somebody to do, and then automate it so it happens by CI automatical.
00:00:55.600 --> 00:01:01.920
The AI Native Dev is a podcast with developers and engineering leads at the cutting edge of AI and agentic coding.
00:01:02.479 --> 00:01:12.079
Join your host, Typer Johnny, and meet Simon Maple every week as we chat with the most exciting voices in AI and tackle the biggest questions that we're facing developers today.
00:01:13.040 --> 00:01:15.280
This episode is the AI Native Dev.
00:01:18.959 --> 00:01:21.439
Hey everyone, hope you're enjoying the episode so far.
00:01:21.920 --> 00:01:29.680
Our team is working really hard behind the scenes to bring you the best guests so we can have the most informative conversations about agentic development.
00:01:30.239 --> 00:01:35.280
Whether that's talking about the latest tools, the most efficient workflows, or defining best practices.
00:01:35.760 --> 00:01:39.200
But for whatever reason, many of you have yet to subscribe to the channel.
00:01:39.599 --> 00:01:46.239
If you're enjoying the podcast and want us to continue to bring you the very best content, then please do us a favor and hit that subscribe button.
00:01:46.480 --> 00:01:53.519
It really does make a difference and lets us continue to improve the quality of our guests and build an even better product for you.
00:01:54.079 --> 00:01:55.519
Alright, back to the episode.
00:01:56.480 --> 00:01:57.840
Hey there, Simon Maple here.
00:01:57.920 --> 00:02:00.879
Welcome to another episode of the AI Native Dev.
00:02:01.519 --> 00:02:06.480
This week we're sharing another fascinating conversation that we had with some of our AI native Devon speakers.
00:02:06.799 --> 00:02:13.280
This time on the topic of AI enablement and what enterprises need to get up to speed with a genetic development.
00:02:13.840 --> 00:02:21.280
I sat down with the godfather of DevOps, Patrick Devoir, Autonomy AIs, Thomas Dubinoff, and Daniel Jones from Resync.
00:02:21.520 --> 00:02:24.240
What I think you'll agree was a really intriguing discussion.
00:02:24.319 --> 00:02:29.199
And what I love is the way that some discussions you very often get question-answer, question-answer.
00:02:29.599 --> 00:02:31.280
But these folks were very, very passionate.
00:02:31.919 --> 00:02:33.680
They really added to each other's points.
00:02:34.000 --> 00:02:35.280
And it was a true conversation.
00:02:35.520 --> 00:02:36.319
I hope you enjoy it.
00:02:37.199 --> 00:02:43.520
Hello, Simon Maple here, and I am at the AI Native DevCon 2026 in London.
00:02:43.680 --> 00:02:53.919
And we've got a wonderful panel here to talk about AI enablement within organizations, how we can be most successful at scaling uh agentic development across our organizations.
00:02:54.000 --> 00:02:55.520
I've got a wonderful panel, as I mentioned.
00:02:55.599 --> 00:02:59.039
We have Daniel Jones, DJ, uh head of product at Resync.
00:02:59.120 --> 00:03:12.000
We have uh uh Tamuz Dubnov, who is the co-founder and CTO of Autonomy AI, and we have Patrick Dubois, who is the DevRel at TESL, the DevOps overlord and godfather of all things pipeline and workflows in DevOps.
00:03:12.240 --> 00:03:13.520
Welcome everyone to the panel.
00:03:13.599 --> 00:03:14.319
How are you all doing?
00:03:14.639 --> 00:03:15.360
Not too bad.
00:03:15.520 --> 00:03:17.520
Given that there was a party on a boat yesterday.
00:03:18.000 --> 00:03:18.319
It was.
00:03:18.560 --> 00:03:20.240
Yes, and I'm just about functioning.
00:03:20.400 --> 00:03:21.120
So that's good.
00:03:21.360 --> 00:03:22.960
Well, that's that's that's always a benefit.
00:03:23.039 --> 00:03:23.599
Always a benefit.
00:03:23.840 --> 00:03:25.120
Even with good weather in England, right?
00:03:25.439 --> 00:03:26.719
Yeah, yeah, even with good weather.
00:03:26.960 --> 00:03:29.680
On a boat on the Thames and it was sunny, who knew?
00:03:29.919 --> 00:03:31.280
Who knew that was a possibility?
00:03:31.520 --> 00:03:34.240
So why don't we start off with uh why don't we start off with you, DJ?
00:03:34.319 --> 00:03:41.199
And we'll let's go left to right and we'll um tell us a little bit about more about yourself, uh your role, and and maybe a little bit about Resync.
00:03:41.520 --> 00:03:41.759
Sure.
00:03:41.840 --> 00:03:45.680
So um I am head of products at Resync, which means productizing our services.
00:03:45.759 --> 00:03:48.639
Uh we are a AI native transformation consultancy.
00:03:48.800 --> 00:03:55.360
We were big in uh the co-founders were big into cloud native transformation, wrote the O'Reilly cloud native transformation book.
Load full transcript
00:03:55.439 --> 00:04:07.280
And what we see is that organizations are repeating the same kind of mistakes when trying to adopt a disruptive technology of just plopping the technology in, not changing anything about how they work and being surprised that they're not going faster.
00:04:07.439 --> 00:04:14.000
So we offer various services uh around that, and I do a lot of the training for uh agentic coding training.
00:04:14.319 --> 00:04:14.639
Awesome.
00:04:15.599 --> 00:04:18.560
Uh so Thomas, co-founder and uh CTO at Autonomy AI.
00:04:18.800 --> 00:04:38.560
Uh, we built an operating system on top of your codebase uh to enable an easy platform for non-technical or semi-technical individuals to actually meaningfully contribute to the code base and the organizations, something we can do their visual iterations and actually get stuff uh ready, do backlog items, do new items, and put everything inside a brown field code base that ends up in a meaningful PR that can actually be merged.
00:04:38.800 --> 00:04:39.759
Sounds good, Patrick?
00:04:40.160 --> 00:04:41.120
Patrick Duel.
00:04:41.279 --> 00:04:47.279
I'm from Belgium and I have gray hair, so I've lived through a bunch of transformations in my life.
00:04:47.439 --> 00:04:52.079
Um most people know me from the DevOps transformation and the cloud native transformation.
00:04:52.319 --> 00:05:00.879
And I love every new chaotic period in the industry because that's where the learning is happening, and right now that is AI and coding.
00:05:00.959 --> 00:05:10.800
And so I'm looking not just at the tools, but also where kind of the new organizational patterns are happening as this field is forming in the industry.
00:05:10.959 --> 00:05:12.319
So that's kind of my gem right now.
00:05:12.560 --> 00:05:15.519
And every time there's a chaotic transformation, the hair gets greyer, right?
00:05:15.759 --> 00:05:18.399
Yeah, it's hard to know whether it's the cause or the effect.
00:05:19.120 --> 00:05:20.000
Like, yeah, I know.
00:05:20.079 --> 00:05:20.879
Brilliant, brilliant.
00:05:21.199 --> 00:05:25.759
So round this table, we're gonna be chatting about all things AI enablement.
00:05:26.160 --> 00:05:43.199
Let's kick off with an interesting question about when we want to scale AI agentic development across our organization in a way where we can control it, it is something we can govern, it is something we can uh do in a meaningful and controlled way.
00:05:43.839 --> 00:05:45.360
Who owns that?
00:05:45.600 --> 00:05:50.959
Which team, which individual, who's responsible for making sure that happens well?
00:05:51.439 --> 00:05:54.480
It's it's a good question and one that I think a lot of people are grappling with.
00:05:54.560 --> 00:06:02.879
And I should imagine that we'll probably end up with uh new departments, new teams, new named units and movements forming.
00:06:02.959 --> 00:06:11.040
Um, I'm working with one customer who happens to have a developer experience team, which is kind of overlaps with the platform team.
00:06:11.199 --> 00:06:21.199
And so there's a an amount of kind of uh it makes sense for those folks to be doing things like building and distributing and assuring the quality of skills, for example.
00:06:21.519 --> 00:06:41.120
Platform teams are kind of interesting in that they're there to help developers by presenting a you know a raised level of abstraction, reduced cognitive load, and you can imagine that like making sure that agentic coding goes well and that you know context is managed and available to agents would fit into uh that group of people's uh responsibility.
00:06:41.360 --> 00:06:50.079
But I've seen other uh customers as well who have a DevOps team, which probably shouldn't get too uh deep into that, whether it's an anti-pattern or not.
00:06:50.160 --> 00:07:00.319
Um but I've seen those folks, like the people that get lumbered with like you're doing the CI CD pipelines for people, like those people also uh being kind of made responsible for these things.
00:07:00.639 --> 00:07:02.160
Patrick, I'd love to go to you there.
00:07:03.199 --> 00:07:08.399
How many hours of sleep did you did you miss when when the DevOps role came about?
00:07:08.959 --> 00:07:09.360
None.
00:07:10.160 --> 00:07:10.480
None.
00:07:11.040 --> 00:07:12.000
No, it's quite easy.
00:07:12.319 --> 00:07:18.480
The big thing is that every new change in an organization usually has a like a small incubation kickstarting team.
00:07:18.800 --> 00:07:20.079
We had the agile team.
00:07:20.240 --> 00:07:22.399
You know, we don't talk about an agile team anymore.
00:07:22.480 --> 00:07:23.680
Like everybody's doing agile.
00:07:23.839 --> 00:07:29.040
We had the DevOps team, and and and that kind of is just a way of scaling things out.
00:07:29.120 --> 00:07:33.439
Like you need that one team that gets duplicated in multiple teams, like, hey, can I repeat this?
00:07:33.600 --> 00:07:34.639
Can I repeat this?
00:07:34.879 --> 00:07:43.839
And then I think the pattern is indeed right now is it falls back to a central team kind of nurturing all the other teams in a way.
00:07:44.079 --> 00:07:51.120
So I think the challenge, what you were saying, like, yeah, it might end up in the developer experience platform team as a role.
00:07:51.279 --> 00:07:56.240
Their challenge right now is they're still often focused on, oh, we'll provide the infrastructure.
00:07:56.800 --> 00:08:00.000
They're not TAI savvy tech coders.
00:08:00.240 --> 00:08:10.639
So that's where there's now a little bit of a vacuum in kind of the companies where, like, okay, you know, they're not the best fit, but they you they do control the spends, they have the models, they have the gateways.
00:08:10.720 --> 00:08:13.199
So you can see their stack kind of expanding.
00:08:13.439 --> 00:08:19.600
Now, whether they will stay in one group as a platform, that's uncertain.
00:08:19.839 --> 00:08:29.680
What you might see is that you're gonna have the cloud
platform team, the AI platform team, the data platform team, and they're all kind of platform supporting this.
00:08:29.759 --> 00:08:34.720
So it's not because there's like one platform group that they have to do it all.
00:08:34.879 --> 00:08:36.960
And that's the same thing with like feature teams.
00:08:37.120 --> 00:08:39.919
They can't do it all because they have a specific focus.
00:08:40.159 --> 00:08:42.639
Small companies, yes, same team, that alliance.
00:08:42.799 --> 00:08:45.840
Bigger companies, they might have like different skills that they're going for.
00:08:46.000 --> 00:08:47.679
So anyway, that's kind of how I look at it.
00:08:47.840 --> 00:08:48.320
Yeah.
00:08:48.559 --> 00:08:58.399
Um as a CTO, you know, you've got your engineering team there, they want to you know use AI as much as possible.
00:08:58.480 --> 00:09:03.200
They don't want to be slowed down through through you know extra governance and things like that.
00:09:03.440 --> 00:09:26.639
Where's the balance between you know having a team wanting to add governance, uh, add ways of working, showing a golden path, versus at this stage of AI's you know, maturity, immaturity, allowing people to kind of find their own paths and understand what's good through developers playing around and trying different things.
00:09:26.960 --> 00:09:27.360
Sure.
00:09:27.840 --> 00:09:31.919
So for us, speed up has been getting the whole team aligned.
00:09:32.080 --> 00:09:40.399
Each person finds a new skill that's important in our workflow in some manner, and then we have a way to share it with all of them and have everybody else say use it, it self-learns.
00:09:40.720 --> 00:09:45.519
Uh, but the mechanism that's really kind of sped up our RD is getting to focus on the stuff that really matters.
00:09:45.679 --> 00:09:52.159
And we get that because we offload the stuff that doesn't need an engineer to our actually our product team and our designer team.
00:09:52.320 --> 00:09:57.840
Uh so part of the game for us has actually been well, we have builders that are engineers, builders that are not engineer.
00:09:58.000 --> 00:10:00.720
How do we manage the non-technical opening the PRs?
00:10:00.799 --> 00:10:04.000
And how do we manage the technical side getting those PRs?
00:10:04.159 --> 00:10:10.320
Uh and that kind of reshuffle is actually that's what has really helped us uh get through that AI native speed up.
00:10:10.559 --> 00:10:16.320
Uh, because you have to do change management internally, both on the receiving end and the the new builders.
00:10:16.720 --> 00:10:22.879
Uh and certainly when that does click, that's where you see everything running forward, and you see the whole team kind of playing together in in harmony.
00:10:23.200 --> 00:10:23.360
Yeah.
00:10:23.600 --> 00:10:35.759
I think that's interesting because you know, when you called in the past DevOps bridging two worlds, like developers and devs coming together, I think the term that like comes up a lot now is the AI product engineer.
00:10:35.919 --> 00:10:39.919
They're like in both worlds trying to cross the silo of the bridge.
00:10:40.080 --> 00:10:43.919
Like you were working on like how to align that better as a field.
00:10:44.000 --> 00:10:54.720
And uh I think that's fascinating in the way that like when the toil or the daily things are being more and more done by AI, we can start like redirecting them.
00:10:54.879 --> 00:10:59.200
Are we it's not about like you know how are we building the thing right, but are we building the right thing?
00:10:59.360 --> 00:11:02.320
And that's kind of where the product focus like comes stronger again.
00:11:02.399 --> 00:11:06.080
And I I think I like kind of how these roles are blending.
00:11:06.159 --> 00:11:11.279
Uh it's not for everybody, I guess, but like it is kind of a good alignment to have in an organization.
00:11:11.919 --> 00:11:17.679
So we see that there's kind of the span of work that needs to be done, and frankly, there's the span of people that care for different parts of the work.
00:11:17.840 --> 00:11:22.639
So there's the stuff that the designers and the PMs really care for, and honestly, that's not the stuff the developers care for.
00:11:22.799 --> 00:11:25.279
So the fact we can say, great, well, those builders are now doing it.
00:11:25.440 --> 00:11:36.399
Hey, developers, UK of our like architecture scaling, like uh the sandbox environments and how they live, you can focus on that because all this uh let's say annoying work you used to have to do.
00:11:36.799 --> 00:11:47.679
Well, now the builders that care about them, they also have the authority to make those decisions, they're the ones doing it, and hey, you can focus on the stuff that you care about and making them uh more agent tech friendly and faster developer.
00:11:48.399 --> 00:11:53.360
That's just happening in our team organically, they're already there, they're already trying to speed up, and they're already figuring it out on their own.
00:11:53.759 --> 00:11:59.200
For me, the focus of CTO is enabling the side that isn't pushing or wasn't pushing as much, which is the non-technicals.
00:11:59.360 --> 00:12:05.679
And when you get them up to speed, suddenly you get the full speed up and you get people working on the stuff that they care about, and then that's the stuff that they're fastest on.
00:12:05.919 --> 00:12:16.000
Yeah, and and it's and speed is the most interesting thing here because I think you know, when we think about a year ago, two years ago, um, gosh, I'm trying to work out how how old uh AI in the mainstream is now.
00:12:16.080 --> 00:12:29.600
But if you think about the last couple of years, we're really thinking about let's just try and get AI usage, let's try and get AI adoption as almost like um you know, get it in the hands of developers, play around with it and see what see what works and what doesn't work.
00:12:29.840 --> 00:12:38.720
It feels like more and more now going
forward, we're thinking about how can we make it work effectively for us and how do we actually get the entire organization using best practices?
00:12:38.879 --> 00:13:07.440
So let's flip it a little bit and say, what are as organizations start rolling out um AI seriously in terms of uh, you know, as a practice across every single uh development team, what would you say the, and I'll ask you this, what would you say are the greatest bottlenecks today, uh, which really maybe it's platform teams, maybe it's other teams, but they're that they're struggling with as part of that scale and that AI enablement across other teams?
00:13:07.679 --> 00:13:13.679
The bottlenecks that that become apparent are often deficiencies in just doing software development well.
00:13:13.840 --> 00:13:23.679
And I I think that we may probably discover in the next couple of years that we spend uh a lot of our time repeating ourselves of the things we've been saying for the last 10-15 years of is your CI CD any good?
00:13:23.840 --> 00:13:24.799
Do you have tests?
00:13:24.879 --> 00:13:27.840
Uh, do you have coding standards that people actually agree on?
00:13:28.000 --> 00:13:31.120
So all of those things uh kind of get exposed.
00:13:31.360 --> 00:13:41.200
And the the uh Dora report showed that if you are not very mature in your software development practices, then agentic coding is likely to make you go slower.
00:13:41.360 --> 00:13:44.399
If you're doing well in development maturity, then you'll go faster.
00:13:44.559 --> 00:13:46.720
The problem is we don't know where that tipping point is.
00:13:46.799 --> 00:13:46.879
Yeah.
00:13:47.039 --> 00:14:03.039
So there are all these things inside the development process, but um, even with successful agentic coding uh adoption, I've seen in a couple of customers and people that uh I've spoken to um on the on the podcast, um uh probably shouldn't plug our own podcast there.
00:14:03.840 --> 00:14:04.240
In the comments.
00:14:04.559 --> 00:14:06.960
You're you're on the R Native Dev, I assumed you were talking about that one.
00:14:07.360 --> 00:14:08.720
Of course you do have your own podcast as well.
00:14:09.120 --> 00:14:09.360
Indeed.
00:14:09.679 --> 00:14:22.399
Um but we've seen uh uh people there of you suddenly speed up uh the act of creating software and then uh products are caught flat-footed, and they're like, Crikey, we didn't have enough stuff in the backlog.
00:14:22.639 --> 00:14:29.600
And that's hard to speed up because it's strategic and you need deep thinking and you need to be talking to customers, understanding their needs.
00:14:29.840 --> 00:14:34.559
So we've definitely seen cases where software development has started to go much more quickly.
00:14:34.720 --> 00:14:40.080
Um, and then in people that do QA as an after-the-fact thing, that's being a bottleneck.
00:14:40.159 --> 00:14:42.159
I'm not sure I'd recommend that pattern generally.
00:14:42.480 --> 00:14:58.639
But yeah, uh product
tends to become uh a bottleneck, and then we need to empower those people to uh to have more free time and to do the strategic thinking and the discovery work that presumably they they really enjoy and they would prefer to be doing rather than writing Jira tickets that developers aren't going to complain about.
00:14:58.879 --> 00:14:59.039
Yeah.
00:14:59.440 --> 00:15:00.399
How about you, Petri?
00:15:01.679 --> 00:15:11.519
I think if you look at the door, you know, first thing is the other option and kind of is it like you know, cranking out new code, that was you know almost like the developer productivity lines of code for a long time.
00:15:11.679 --> 00:15:14.000
Then we learned, okay, it needs to work in production.
00:15:14.399 --> 00:15:19.840
So we brought in the metric actually, like how many times do we need to rework it if it isn't good?
00:15:19.919 --> 00:15:22.399
Like that kind of is your defect rate, almost like that.
00:15:22.480 --> 00:15:30.000
You kind of need to be on control both in generating and kind of making sure the defects are not like are in balance there as well.
00:15:30.240 --> 00:15:53.360
And I think I what I like as an uh as an almost like new um proxy metric that like when you look at like not just the users coding with AI, but if the agents are doing the coding, you look at uh the things that like how many how many uh almost like turns does an agent need to do to do their job effectively?
00:15:53.679 --> 00:16:02.639
And you kind of can get that number down by you know, you can select another model, you can kind of give it the right tools, and you can provide it better context.
00:16:02.879 --> 00:16:11.600
So in that way, it is not anymore like can I do better code in production, but can my agents do better code towards production?
00:16:11.759 --> 00:16:13.440
So it becomes like delegated.
00:16:13.679 --> 00:16:25.440
So you need to think out how effective they are, and that's kind of a flip that I see is now slowly moving from the def working with AI to the def instructing the agents to use AI to do it.
00:16:25.519 --> 00:16:29.200
And that that's kind of like an interesting proxy metric, I think, to track.
00:16:29.279 --> 00:16:49.759
Uh and I think there's something both amusing and mildly depressing about this in that when we have uh software factories and that idea matures and becomes more commonplace, we're gonna have this infinitely tunable and measurable way of developing software, and we see exactly how effective it is in that way that you mentioned of looking at how many turns are used, how many tokens are used.
00:16:49.919 --> 00:16:52.720
And unlike with human teams, we're gonna be able to A-B test.
00:16:52.960 --> 00:17:03.120
We could like have uh prompts and software factory setups that you know use one approach versus another, and we can reset their memories and try it a second time and see did that work better, did it not?
00:17:03.200 --> 00:17:08.400
And you can't really do that
with software engineers and real teams because you can't like do a men in black and memory wipe them.
00:17:08.559 --> 00:17:26.000
Um but the thing that's like amusing and depressing is um that now that it's agents doing the work and tokens and the cost of those, everyone's gonna be really like keen on making sure the software development uh lifecycle was as effective and as efficient as it possibly could be.
00:17:26.160 --> 00:17:30.720
But when it was humans, it was like, oh well, Timmy's just lazy and that's what that's why stuff isn't getting delivered.
00:17:30.880 --> 00:17:37.119
And they didn't focus enough on process and you know value stream mapping and all those important things to tune it.
00:17:37.200 --> 00:17:38.880
It was just, oh, the humans can suffer.
00:17:38.960 --> 00:17:45.119
But now it's when it's agents, it's like, well, that's gonna cost us money, and we can easily see that it's costing us money, so now all of a sudden we care.
00:17:45.359 --> 00:17:45.519
Yeah.
00:17:46.160 --> 00:17:53.039
It's funny we face this challenge today because we we have agents working on 160 plus organizations, which widely different code bases.
00:17:53.359 --> 00:17:57.920
And we see the ones that are are better quality, the agent does a better job faster.
00:17:58.240 --> 00:18:04.640
The ones that are lower quality, we have an onboarding flow that the agents get better after a few tasks, they onboard themselves to your repo.
00:18:04.799 --> 00:18:08.880
And we see that in the difficult projects, it takes them longer to onboard to good quality.
00:18:09.519 --> 00:18:13.279
But we cannot go to we we focus on onboarding quickly to an organization.
00:18:13.680 --> 00:18:24.640
We go top-down, so that means the the business leaders we talk to, we cannot say, hey, like a sprint to refactor so the agents can have a better time, so everything will go smoother because they don't really care about that.
00:18:24.720 --> 00:18:26.319
They just care about speed, like you mentioned.
00:18:26.720 --> 00:18:36.319
So for us, it's been all about how do you mix in doing the user-facing work quickly while writing the natural code churn that the organization has to slowly refactor.
00:18:36.480 --> 00:18:38.720
But we have like a we call it the PAP framework.
00:18:39.279 --> 00:18:44.960
And as you do work, you you slowly refactor bit by bit, so it's easier for the next agent that comes along in that part of the code.
00:18:45.279 --> 00:18:50.400
But you do not come and say, hey, dev team, you have this task, make it easy for our agents to work there.
00:18:50.559 --> 00:18:57.440
You have the agents doing the work in bite-size going back to the PR fatigue in a way that doesn't inflate the PRs because then people won't go over them.
00:18:57.599 --> 00:19:00.160
So it's it's a really it's a balancing game.
00:19:00.400 --> 00:19:15.279
Getting the leadership happy because features are being delivered, getting the developers happy because PRs are not too big, and getting the non-technical PMs, the designers happy because they can do the work, all while slowly pushing your agenda better and better coding standards.
00:19:15.680 --> 00:19:18.720
So everything needs to happen uh kind of weaved together.
00:19:19.200 --> 00:19:29.359
And I'd love to kind of while while while talking about that, you you previously you mentioned about the kind of like the the technical devs, the traditional devs, and the non-technical, so maybe it's a PM or something like that.
00:19:29.599 --> 00:19:40.640
When we talk about rolling out an enablement here, do you see different types tend to see different bottlenecks between a technical dev and a non-technical dev when adopting like a more of an agentic process there?
00:19:42.000 --> 00:19:42.640
Uh yes.
00:19:42.880 --> 00:19:44.480
Okay, so we'll go with the technical side.
00:19:44.960 --> 00:19:47.680
Uh the technical side, there's an ego that we all know.
00:19:48.400 --> 00:19:52.559
Uh we are working through that, whether it's with clients or whether it's internally, like the devs.
00:19:53.119 --> 00:19:57.200
Uh so for us, they they interface with us, they can use our platform.
00:19:57.440 --> 00:20:02.960
Uh not not to show off, but like the PRs that come out of them versus out of the platform versus developers.
00:20:03.200 --> 00:20:11.599
Often our platforms they do a better job because they map your your coding practices much more in depth than developers and they keep much better tracks of existing uh components.
00:20:11.759 --> 00:20:16.000
Like we have clients with 2700 components, no developer will know which component to use.
00:20:16.240 --> 00:20:21.680
We do know we do data science on it, and and we can actually re-rank and pick between them in a much more intelligent way.
00:20:22.000 --> 00:20:25.359
But you have to come over that ego of the developer viewing the PR.
00:20:25.759 --> 00:20:26.559
So we tuned it.
00:20:26.640 --> 00:20:28.400
We're a little gentle when it comes to them.
00:20:28.720 --> 00:20:31.359
Uh and then they're very fragile online developers.
00:20:31.599 --> 00:20:32.720
I I'm not gonna say that.
00:20:34.400 --> 00:20:35.279
Did the content say that?
00:20:35.359 --> 00:20:35.759
I can't remember.
00:20:37.200 --> 00:20:39.759
Uh and then on the PM side, it's a confidence thing.
00:20:40.000 --> 00:20:43.279
It's like, hey, I'm I'm okay opening a PR.
00:20:43.759 --> 00:20:47.200
Uh it's getting the dev team to play along.
00:20:47.359 --> 00:20:52.480
We we have some experiences where uh PMs open PRs and the dev team pushed back.
00:20:52.720 --> 00:20:55.519
And then we asked like the team lead, look at the code, is it good?
00:20:55.599 --> 00:20:56.960
It's like, oh, this is solid code.
00:20:57.200 --> 00:20:58.559
Why did the developer push back?
00:20:59.039 --> 00:21:00.480
Because it didn't come from a developer.
00:21:01.039 --> 00:21:07.359
Uh so when that's worked out, we basically work on a
conference of the PMs saying, hey, you don't need to write a ticket, that's not the deliverable.
00:21:07.680 --> 00:21:14.079
You don't need to make an HTML prototype that is disconnected, that is not the deliverable, that's effectively a ticket in a Figma.
00:21:14.640 --> 00:21:17.039
Uh you need to actually prototype inside the code base.
00:21:17.119 --> 00:21:18.799
You need to edit existing stuff in the code base.
00:21:18.960 --> 00:21:22.160
Hey, it's possible, hey, it's feasible, hey, effectively it's really easy.
00:21:22.240 --> 00:21:27.279
And when you work with the agent, it already knows the constraints from the code base built in when you interact with it.
00:21:27.680 --> 00:21:33.359
And yes, when you click send to dev and it goes to the developers as a PR, be proud and don't be scared.
00:21:33.599 --> 00:21:35.359
Uh and that's like a big enabler.
00:21:35.680 --> 00:21:46.160
It's almost like an imposter syndrome, I guess, for non-technical folks to feel that maybe they're resistant to do things with confidence because they feel like they're not a dev and the devs gonna scrutinize it more almost.
00:21:46.559 --> 00:21:47.119
They're scared.
00:21:47.200 --> 00:21:48.720
They don't know how to answer technical questions.
00:21:48.799 --> 00:21:49.200
Yeah.
00:21:49.680 --> 00:21:51.920
And that's a part of the process we're we're going through.
00:21:52.160 --> 00:21:53.200
Like, you don't need to.
00:21:53.279 --> 00:21:54.319
That's the beauty of the agents.
00:21:54.400 --> 00:21:56.480
They know the code base, they manage it for you.
00:21:56.640 --> 00:21:59.519
You need to give us the product intent, and that's what matters.
00:22:00.319 --> 00:22:05.680
The agent delivers something visual, then it's a part of their app that is rendering and running and they can interact with it.
00:22:05.759 --> 00:22:10.720
If you do the product review or the design review, depending on your persona, you should be confident.
00:22:10.799 --> 00:22:15.119
The code works the way you want it to, the code quality is there, you don't need to worry about it.
00:22:15.279 --> 00:22:18.880
And hey, if there's like chess run calls on the back end side, the developers will take it.
00:22:20.240 --> 00:22:38.559
So let's move in a different direction now where we're going to talk about ways in which agentic development, or rather, patterns of agentic development, how people are doing this in the real world and what we see from people externally and in what they're saying they're doing and how they're approaching it.
00:22:39.039 --> 00:22:43.039
Patrick, you recently created a really interesting tool.
00:22:43.279 --> 00:22:48.559
It's called the TESL Patterns site, Tesla.io forward slash patterns.
00:22:48.960 --> 00:22:51.200
And it does a ton of research in the background.
00:22:51.279 --> 00:22:55.039
I think it uses the Carpathy wiki approach in the background.
00:22:55.200 --> 00:23:17.599
And it provides a bunch of patterns that people can look at and say, ah, okay, I can see that people are thinking and doing things in this way, this approach, and you know, cites various uh social uh, you know, I guess uh uh ways in which people are going into more depth to say, yes, I I did it like this on on in this infrastructure in this way.
00:23:17.839 --> 00:23:22.160
Talk us through a little bit about that and maybe highlight a couple of the more interesting patterns that you found.
00:23:22.559 --> 00:23:22.799
Yeah.
00:23:23.119 --> 00:23:28.480
So I think one of the reasons is that when you do surveys right now, you get like an enormous spread.
00:23:28.559 --> 00:23:35.359
Like, you know, person A is savvy, the other ones are not savvy, and then the effectiveness and yes and no, that's hot.
00:23:35.599 --> 00:23:43.279
So I figured that what if we look at the socials on how they do things and kind of capture that signal.
00:23:43.440 --> 00:23:44.720
No, that has a little bit of a problem.
00:23:44.880 --> 00:23:48.480
It could be vendors saying how they do great things and practitioners.
00:23:48.720 --> 00:23:58.319
But kind of the whole idea is that we filter out like how how some of the people are using this, and you know, if there's many people doing similar things, that boils down to kind of a pattern.
00:23:58.640 --> 00:24:07.519
Now, one of the surprising and unsurprising things is almost like what is good for a eye is actually good for a human and the other way around as well.
00:24:08.000 --> 00:24:18.000
Like given that you're working with an agent, um, if you actually give your agent good documentation, lo and behold, it performs better.
00:24:18.240 --> 00:24:23.200
If you give them tests, then you actually know whether that's still working, yes or no.
00:24:23.359 --> 00:24:26.799
Uh so if you have observability, it can expect itself.
00:24:26.960 --> 00:24:38.720
So that's kind of like you know, when you start thinking about like these analogies on like what is actually good and what is not, you can go up to um rituals, for example, on the team.
00:24:38.799 --> 00:24:44.880
Uh, we have a team lead and they're used to doing scrum and retros and kind of those things.
00:24:45.680 --> 00:24:52.400
But what if you would have an agent review kind of at the end of the sprint?
00:24:52.559 --> 00:24:53.599
What were they missing?
00:24:53.759 --> 00:24:55.279
How effective were they are?
00:24:55.519 --> 00:24:57.839
But it's not also what do you do as a team lead?
00:24:57.920 --> 00:24:59.680
You set a goal for a person.
00:24:59.839 --> 00:25:02.240
So your goal sets the measurement.
00:25:02.400 --> 00:25:04.000
So that is another account.
00:25:04.079 --> 00:25:16.960
So you find like people doing like those little ceremonies, or for example, they do uh writing um almost like context together as a pair, so you get nuances.
00:25:17.039 --> 00:25:19.359
So that's the pair programming, but now with context.
00:25:19.440 --> 00:25:24.160
So that's kind of interesting
that those signals are bubbling up uh in a certain way.
00:25:24.400 --> 00:25:29.119
Which is good because it actually goes back to what you said earlier about it's good software development practices.
00:25:29.359 --> 00:25:38.720
Everything that you mentioned there should be things that we are doing today manually, but it's about almost like can I say the word agentification?
00:25:38.960 --> 00:25:42.559
Uh you know, it's essentially making an agent take care of all of that.
00:25:42.880 --> 00:25:43.279
Yeah, yeah.
00:25:43.440 --> 00:25:46.559
But I I think it's uh, and you already mentioned it.
00:25:46.640 --> 00:25:58.160
So if you want yes, it gets used, uh it gets a little bit getting used to as you know, what's the new world as a PM and an engineer, and kind of I don't I feel okay contributing and working in that.
00:25:58.319 --> 00:26:08.960
But if for example, if there's one advice that I would give a developer right now, moving to a gentic, is just the most important thing would be don't repeat yourself.
00:26:09.599 --> 00:26:14.480
Like don't keep telling the agent what to do, but write something down.
00:26:14.640 --> 00:26:19.039
Uh if you're using a tool to verify what it's doing, give it the tool.
00:26:19.119 --> 00:26:24.480
So you're not having so kind of that is a very strong, like almost like mindset.
00:26:24.720 --> 00:26:27.359
Uh and it's different from oh, let's collaborate.
00:26:27.519 --> 00:26:31.680
Like, not not collaborate, like just have it do all the work.
00:26:31.759 --> 00:26:35.519
And and that pressures you in all the correct engineering points.
00:26:35.599 --> 00:26:50.720
And I think when AI encoding came up, oh, you know, the typical is like uh this is the end of coding and there's no more engineering, and lo and behold, you know, if you really get want to get all the value of that, it is good engineering that we bring in there.
00:26:50.799 --> 00:26:57.519
So and the engineering becomes creating the the software factory or the assembly line or whatever you want to call it.
00:26:57.759 --> 00:27:12.400
I can imagine that for a lot of like I can see a bifurcation for developers of the more product-minded ones who like achieving outcomes and getting motivated by I delivered a feature to users they find valuable, they're gonna kind of become more product uh focused.
00:27:12.480 --> 00:27:23.119
And the ones who are like, I like making nice, teeth uh tidy, neat code, they're going to be more motivated by I'm going to make the best agentic software uh process that I can.
00:27:23.200 --> 00:27:28.000
I'm gonna tweak the the dials and the knobs on how the software gets generated.
00:27:28.240 --> 00:27:31.200
So that kind of engineering mindset doesn't go away.
00:27:31.279 --> 00:27:35.119
It's just you're not applying it to the code, you're applying it to the machine that makes the code.
00:27:35.440 --> 00:27:35.839
Exactly.
00:27:36.000 --> 00:27:36.559
Yeah, yeah.
00:27:36.720 --> 00:27:46.799
And I I think that is interesting about um we had that when we went to the cloud uh the cloud, everybody wants to like build their or rebuild their own kernel.
00:27:46.960 --> 00:27:47.519
Like, why?
00:27:47.680 --> 00:27:48.799
Like it's it's good enough.
00:27:48.960 --> 00:27:51.279
We we don't need everybody to build their own kernel.
00:27:51.440 --> 00:28:01.440
So if there was one advice that uh you know, people come up to me uh you know over all the years of DevOps and I say, but yeah, you know what, we're special, it doesn't work for us, blah, blah, blah.
00:28:01.599 --> 00:28:01.920
Guess what?
00:28:02.000 --> 00:28:06.000
Like, you know, the industry kind of said, like, that loop is actually universal.
00:28:06.319 --> 00:28:09.359
It can work everywhere within software development.
00:28:09.519 --> 00:28:27.359
And so my advice then to the platform team would be saying, like, okay, your number one focus is telling people that like they're not so special in what they're building for code, uh, but it's their mindset that is the difference in kind of how they approach the problems and what they bring into that mix.
00:28:27.519 --> 00:28:30.799
And it's it's really weird because everybody's repeating this, which is great.
00:28:30.960 --> 00:28:36.880
Like every new technology has a learning phase, and we all have to go through the motion and kind of say, hey, oh yeah, now I understand.
00:28:36.960 --> 00:28:44.400
And and yes, all the stories about like I vibe coded this app, like all great, but how do we bring this into the maturity of an organization?
00:28:44.559 --> 00:28:51.039
And that's only by saying, like, well, we're gonna probably don't need to, every developer needs to build their own pipeline.
00:28:51.119 --> 00:28:52.640
That's probably not very effective.
00:28:52.720 --> 00:28:57.359
And oh, it's actually the same thing over and over again with two variables or something.
00:28:57.519 --> 00:29:02.480
And that's kind of the mindset of the platform, the reusability across different teams that goes in.
00:29:02.799 --> 00:29:09.200
I can share a little bit about our process internally, and I guess like the RD leadership perspective I try and push.
00:29:09.519 --> 00:29:15.680
Um, the way I think about it is I go back to pre-gen AI and I imagine I had all the manpower and all the time in the world.
00:29:15.839 --> 00:29:18.960
Somebody opens a PR, I would want to map out is it high risk, is it low risk?
00:29:19.200 --> 00:29:19.920
PR is merged.
00:29:20.079 --> 00:29:25.440
I'd wait two weeks, I want to go over the logs and see did what they do, was it successful, was it not?
00:29:25.519 --> 00:29:27.200
Is there some edge case they didn't cover?
00:29:27.359 --> 00:29:31.519
Uh and basically with that mindset, we say, okay, now you have Gen AI, you have agents.
00:29:31.680 --> 00:29:37.839
There's no excuse not to do everything you ever thought you'd want somebody to do, and then automate it so it happens by CI
automatically.
00:29:38.000 --> 00:29:43.440
Uh so for example, here, all of our PRs get labeled by risk automatically by an agent.
00:29:43.680 --> 00:29:44.079
Cool.
00:29:44.319 --> 00:29:47.440
Uh then they go to staging, they live in staging during the QA process.
00:29:47.680 --> 00:29:53.119
Before we release, we have another agent that goes PRPR, is sorted by the risk level, and goes through our logs.
00:29:53.200 --> 00:29:57.279
Our logs are queryable for agents, and they actually go and check every single PR.
00:29:57.440 --> 00:29:58.400
Hey, what was the intent?
00:29:58.559 --> 00:29:59.920
Hey, what did the logs look like?
00:30:00.079 --> 00:30:02.079
Did it cover what it was supposed to change or fix?
00:30:02.240 --> 00:30:03.920
Was there some edge case we missed?
00:30:04.319 --> 00:30:06.880
And it's been amazing that it's caught more and more stuff.
00:30:06.960 --> 00:30:16.559
Like the high-risk PRs often do miss something, even though our engineers are agentic in how they work and everything, and we have QA and we have testing and we put in all this effort.
00:30:16.720 --> 00:30:20.720
But hey, when you retro it a week after it's in staging, you find more stuff.
00:30:20.880 --> 00:30:22.640
Uh and often it's like easy stuff.
00:30:22.720 --> 00:30:29.119
Like the agent knocks it out, it's like, oh, this edge case pops up once and never because we're a non-deterministic product because of Gen AI and LLMs.
00:30:29.359 --> 00:30:31.680
But hey, now we see that and let me capture it too.
00:30:31.759 --> 00:30:36.480
Uh so that sort of mindset of if I had all the manpower in the world, what would I tell them to do?
00:30:36.720 --> 00:30:39.759
Okay, well, start having agents do it once, twice, automate it.
00:30:39.839 --> 00:30:39.920
Yeah.
00:30:40.079 --> 00:30:44.720
For us, we put it into our release cadence, uh, and suddenly you just get better and better quality.
00:30:44.799 --> 00:30:49.920
And you move stuff that you would need to think about, or your developers would need to think of, just move it.
00:30:50.079 --> 00:30:50.720
It's just automatic.
00:30:51.119 --> 00:30:57.920
I love that you bring up the risk thing because there's the belief that the digital factories, you know, kind of will churn automatically and all the things.
00:30:58.079 --> 00:31:08.000
But because when you know what the risk is, uh, you know, again, translating it to a management world, different risk levels require different management techniques.
00:31:08.160 --> 00:31:09.759
Like, are you micromanagering?
00:31:09.920 --> 00:31:13.359
Yes, probably the risk is very high, and then you need to do that.
00:31:13.519 --> 00:31:23.039
If you have guardrails or something that is not that like important, or you have like some way of mitigating this in the product, by all means, then we'll get the feedback later.
00:31:23.200 --> 00:31:31.039
So that is the the levels and that you can only do that if you start like thinking about like uh risk levels of like things pushing through.
00:31:31.359 --> 00:31:33.440
We even couple it to our uh PR fatigue.
00:31:33.519 --> 00:31:38.000
So high risk, get more human resources to go over it, and we even couple it to our token budget.
00:31:38.079 --> 00:31:42.240
So low risk, we're not gonna spend a lot of effort to check it from an agentic LM spend.
00:31:42.319 --> 00:31:43.839
Uh so just a way to prioritize everything.
00:31:44.160 --> 00:31:45.839
Sounds a lot like running a business now, right?
00:31:47.440 --> 00:32:03.519
When you talk about adding this into like you know, instrumenting this through your CI, is CI still fit for purpose in this in this flow, or are there alternatives or or or changes we need to make to our CI style process to make it better for a genetic workflows?
00:32:03.839 --> 00:32:06.160
Oh, there's like key transformations you have to do.
00:32:06.480 --> 00:32:20.160
So one key transformation is whatever your observability platform is, you need to make sure it's easily queryable for an agent so it can verify both while it's doing a task and also retroactively an entire like I don't know, dev environment, staging environment, prod environment, you need to make it really easy for the agent.
00:32:20.400 --> 00:32:27.039
And we have skills that also tune themselves so we continuously get better, especially as our product evolves and our logs change and the way you query change.
00:32:27.279 --> 00:32:28.799
So that's like a key part.
00:32:29.119 --> 00:32:32.960
Uh RCI is very agentic heavy.
00:32:33.519 --> 00:32:37.839
Uh even the way you push and open a PR is extremely agentic heavy.
00:32:38.319 --> 00:32:45.839
Uh and all the actions that we put in from a DevOps perspective are also like really agentic heavy.
00:32:46.160 --> 00:32:53.039
Uh that's just a way not to have uh like you want into end, you want unit test, but you want those all those mechanisms for feedback to the agent.
00:32:53.279 --> 00:32:55.039
So the agent works in higher confidence.
00:32:55.279 --> 00:32:59.680
Uh but you gotta think of as the agent stepping in, and how are they doing more than just writing code?
00:32:59.839 --> 00:33:05.279
How are they your confidence layer that there is quality, that the risk is managed, uh, that stuff is moving faster?
00:33:05.359 --> 00:33:10.319
We even have agents help us prioritize, like I don't know, you have 20 PRs open, which one is high importance?
00:33:10.400 --> 00:33:12.960
The agents will tell us and push the reviewer to act on it.
00:33:13.440 --> 00:33:31.119
And is it do you do you see it almost like as this uh um this you know series of actions that need to happen, or do you feel like it's you know more fluid than that where you have you have different different agents providing you with different data to make you know decisions more integrated, more collaboratively?
00:33:31.839 --> 00:33:36.319
Okay, so I'm gonna go with your answer, which is it's fluid until I figure it out.
00:33:36.480 --> 00:33:36.559
Yeah.
00:33:36.720 --> 00:33:38.480
And then it's hard-coded because I don't want to repeat it.
00:33:38.559 --> 00:33:39.759
I want to get that mental load off.
00:33:40.160 --> 00:33:58.240
I've I've I've got um a strong hunch that um or strong opinions really on this in that um when when you're a software developer and you're trying to work out, like you're implementing a feature, like one, just make it work, then make it work maintainably, then make it work readably, performantly, securely.
00:33:58.400 --> 00:34:02.079
You've got all these different lenses through which you have to look at a code change.
00:34:02.720 --> 00:34:09.199
I think that what we need um with agentic development is to apply all of those lenses deterministically.
00:34:09.360 --> 00:34:15.519
All right, instead of hoping that an agent is going to figure out all of this in one pass by sticking all the things in agent's MD.
00:34:15.760 --> 00:34:18.400
Instead, we can have some kind of CI-like process.
00:34:18.480 --> 00:34:33.920
I've built a little tool called Assembly Line that does this where you you make a change with your agent or the agent makes a change by itself, and then another agent gets spun up with one deterministically defined prompt of check for dead code, check for missing test coverage, check for this, check for that, check for the other.
00:34:34.000 --> 00:34:36.480
And to your point, you've got infinite engineering resource.
00:34:36.559 --> 00:34:50.239
So why wouldn't you look at a change through all of those lenses and then try and provide feedback as fast as possible to the agent that was doing the main kind of set of changes so it doesn't deviate too far and uh can fold that back in.
00:34:50.559 --> 00:35:04.480
So I think the like we need to, as an industry or you know, as practitioners, figure out that the flexibility and power of LLMs to work in a non-deterministic fashion is really powerful, but we don't want that all the time.
00:35:04.639 --> 00:35:11.679
I know of some people that are like using agent skills to figure out how to deploy stuff into prod, and I'm like, just build a platform.
00:35:11.840 --> 00:35:16.079
Like we've had platforms for 10 years, they're nice and deterministic and straightforward.
00:35:16.320 --> 00:35:22.960
Maybe use agents to build your platform, but don't replace your platform with agents burning tokens reinventing the wheel.
00:35:23.119 --> 00:35:36.159
Like there's a place for the um determinism, and I think it is in that kind of figuring out, like you explore and then you figure out okay, right, that was the right place where we should have run that kind of check or uh a pass or sweep over the code base through this lens.
00:35:37.519 --> 00:35:39.039
I I have a ton of ton of say.
00:35:39.199 --> 00:35:39.760
Okay in a second.
00:35:39.920 --> 00:35:40.480
So excuse me.
00:35:40.639 --> 00:35:42.559
Uh so we have two harnesses in my organization.
00:35:42.639 --> 00:35:45.440
We have the harness in the product and the harness for the RD team internally.
00:35:45.679 --> 00:35:52.000
Uh so it's beautiful because what you said is stuff we have in both harnesses, meaning we do not want the agent to go through this route to exactly what we want.
00:35:52.159 --> 00:35:57.199
We want the agent to do a big step this way, and then the one that step makes sense for it statistically.
00:35:57.519 --> 00:36:00.719
And then a big step this way, again, it makes much more sense to it statistically.
00:36:00.960 --> 00:36:03.039
You see that you're no longer fighting with the LLM.
00:36:03.119 --> 00:36:05.440
It does not want to do one, two, three steps in one.
00:36:05.519 --> 00:36:07.840
It wants to do one, wants to do two, wants to do three.
00:36:08.000 --> 00:36:09.599
You just get better quality that way.
00:36:09.760 --> 00:36:19.920
And it's funny you call it assembly line, we call it nudging, uh, but in that sense, you get a very clear workflow uh and you no longer fight the the agent and how it wants to work.
00:36:20.159 --> 00:36:30.639
Uh what we do internally for our like RD team, the RD harness, not the product, uh, is sadly we have to factor in the human element in in just because we do.
00:36:33.360 --> 00:36:34.960
So we developed the humans.
00:36:35.760 --> 00:36:37.280
It's like the slowest part of the whole machine.
00:36:37.760 --> 00:36:37.920
It is.
00:36:38.159 --> 00:36:43.039
Uh so we developed kind of pump, is what I call it, um, which is plan merge polish.
00:36:43.280 --> 00:36:46.079
The code is moving so fast you can't have an open PR.
00:36:46.400 --> 00:36:52.639
Meaning, if it's an open PR and I want to do QA on it, and I want to do like a product review, and I want to do a design review, that is all lovely.
00:36:52.800 --> 00:36:57.840
By the time those are done, the code is gonna have so much conflict, everything looks different, I have to restart.
00:36:58.239 --> 00:37:03.760
I I I am gonna be so, so happy if we manage to go back to trunk-based development as an industry.
00:37:04.000 --> 00:37:08.880
Like pull request-based workflows inside enterprises are a damn silly idea.
00:37:08.960 --> 00:37:16.639
Um they make great sense in open source repositories with people who are not strategically aligned and you you don't trust, and you know, it makes sense there.
00:37:16.800 --> 00:37:24.239
But like that, because of that speed change that you mentioned, like you've got to get the feedback to the agents working on the code as quickly as possible.
00:37:24.559 --> 00:37:34.559
And that kind of goes back to a point that I think is running through all of this, that the fundamentals of like what made good software delivery, what made good uh methodology, those haven't changed.
00:37:34.719 --> 00:37:38.320
Things like fast feedback are uh important, they always will be.
00:37:38.400 --> 00:37:53.280
Having feedback, being able to tell, like you know, the point about observability,
that the agent can tell whether it's broken something in production, having all those feedback signals is was important before for humans and continues to be important for agents, if not being even more important because of the rate of change.
00:37:54.480 --> 00:37:56.000
So I'll expand 100% agree.
00:37:56.559 --> 00:38:02.239
Uh the the issue there is also that as soon as a PR is merged, the the landscape changed, the code changed.
00:38:02.400 --> 00:38:04.639
So the next agent that starts will do a different job.
00:38:04.880 --> 00:38:08.159
Like everything impacts it, the statistics of the path the LM goes through.
00:38:08.400 --> 00:38:11.519
Uh so for us we we called it like the the M in the pump, the merge.
00:38:11.920 --> 00:38:14.239
As soon as the PR is open, we want to get it merged as fast as possible.
00:38:14.320 --> 00:38:16.320
So hey, human reviewer, look at it right away.
00:38:16.559 --> 00:38:17.039
Get it in.
00:38:17.119 --> 00:38:18.239
We need it to pass our CI.
00:38:18.320 --> 00:38:19.599
We do not need it to pass QA.
00:38:19.760 --> 00:38:23.760
We do not need to pass the product or the designer, everything is feature flagged.
00:38:24.000 --> 00:38:25.679
Then we go into the last P, which is polish.
00:38:26.079 --> 00:38:29.840
Stuff is in the code, every new feature that gets developed is off of the same code.
00:38:30.079 --> 00:38:41.119
And our product manager can go and review it and they can polish it using our platform and get the UI and UX exactly right and see all the bad decisions from the user-facing perspective that the developer made.
00:38:41.360 --> 00:38:44.480
Well, the developer got the feature working, it did not get the feature polished.
00:38:44.639 --> 00:38:45.199
So that's fine.
00:38:45.360 --> 00:38:46.559
We have other personas for it.
00:38:46.719 --> 00:38:53.360
And now they can do their work in a way that is smooth within the evolving code base, like it's evolving really quickly.
00:38:53.840 --> 00:38:59.599
Uh so that kind of feature flagging, merging quickly with feature flags enables us to do it kind of two-step.
00:38:59.840 --> 00:39:00.639
There's no one PR.
00:39:00.800 --> 00:39:15.039
A feature should take three or four PRs, one PR from the developer, one PR from the product manager that changes functionality, one PR from the designer that tunes the UIUEX, and then maybe one last PR from the QA that saw some sort of itch case.
00:39:16.320 --> 00:39:24.559
And in terms of like the the human reviewer being the slowdown there, I had a really interesting chat with uh Ryan Lapopoli, who's a member of technical staff at uh OpenAI.
00:39:24.800 --> 00:39:31.039
And um one of the things that they're doing in some of their projects is to entirely remove uh a human reviewer.
00:39:31.199 --> 00:39:34.000
Uh, how much does that make you go, oh gosh, that's horrible?
00:39:34.159 --> 00:39:49.119
Or how much does that make you think, oh wow, there's a real opportunity here in certain cases for us to actually allow either an AI or tooling to perform those uh reviews and give us that feedback, maybe for different levels of criticality of change.
00:39:49.199 --> 00:39:51.199
But is that something that you look at, Mike, much?
00:39:51.599 --> 00:39:52.159
Oh, all the time.
00:39:52.320 --> 00:39:52.400
Yeah.
00:39:52.559 --> 00:39:54.800
So again, it goes back to the risk uh conversation.
00:39:55.039 --> 00:39:56.239
High risk, no way.
00:39:56.639 --> 00:39:57.840
Uh I want a human person.
00:39:57.920 --> 00:40:02.000
And I also define like risk is also by what part of the code base they're touching, how sensitive it is.
00:40:02.159 --> 00:40:14.239
Uh but beyond it, I'm trying to automate the PR commenting and comment resolution process, uh, which means I have uh agents that have modeled every single developer on my team and their comments for the last few hundred PRs.
00:40:14.639 --> 00:40:16.159
And I know each and every persona.
00:40:16.800 --> 00:40:22.480
So as soon as a PR is open, we have a whole prepare PR, which is a bunch of agents to check it in all the ways that we've modeled it, we need to check it.
00:40:22.719 --> 00:40:31.679
Once a PR is open, we launch different agents that model different people from our team and comment as if they were those people, and does a surprisingly good job.
00:40:31.920 --> 00:40:34.239
Then we trigger agents again to resolve those comments.
00:40:34.559 --> 00:40:39.920
Then we trigger the retro to see are there any error logs, stuff actually behaves like it wanted to.
00:40:40.159 --> 00:40:47.440
And again, if the risk is not high from the PR's nature, I'm moving towards being able to like skip the human reviewer at all.
00:40:48.239 --> 00:40:49.840
Again, I have developers on my team.
00:40:50.000 --> 00:40:54.719
I have some developers on my team that are absolutely against this because they want the developer to own.
00:40:54.880 --> 00:40:58.480
So I'm going through the motion of massaging them and getting them comfortable with the idea.
00:40:58.559 --> 00:41:00.239
Uh but I'm definitely pushing for that.
00:41:00.559 --> 00:41:04.559
And it depends how reversible it is as well, in terms of uh It's the cardrails, right?
00:41:04.639 --> 00:41:14.320
And it was the same thing, like when everything was being automated during the DevOps transformation, people said, like, but if I did change one thing, I can delete everything now, automate it, right?
00:41:14.559 --> 00:41:30.639
But it was the harness of test the CR CD that kind of was like the fail system of, and then the narrative was you can have a junior come in, push to your uh you know, kind of your main, and like your harness, your test harness will catch it.
00:41:30.719 --> 00:41:32.320
Like if that's a good one.
00:41:32.960 --> 00:41:43.519
Uh and but what you saw is that there was still a difference between I'm gonna do changes in the code of the compute to I'm gonna do changes on the data scheme
model.
00:41:43.679 --> 00:41:46.239
Because people felt the risk is high.
00:41:46.480 --> 00:41:50.800
So you you can see kind of similar patterns on like uh the choices that you make.
00:41:50.880 --> 00:41:53.920
Uh but it's it's a mindset, like how far do you go?
00:41:54.079 --> 00:41:56.480
There's also a cost to kind of doing the automation.
00:41:56.559 --> 00:42:00.159
And you mentioned a lot of things like, yeah, I have an agent for this, for that.
00:42:00.320 --> 00:42:06.159
Uh, you know, putting everything in place is it's it's currently like a lot of work, right?
00:42:06.239 --> 00:42:18.880
Like and yes, you can buy your product or your product or like everybody's product, but that that kind of is uh, you know, funnily enough, we hoped everything was be more simple, five-coded, and everything were working.
00:42:19.039 --> 00:42:32.719
And what we end up, and that's usual for any technology maturing, is that when it's matured, it's even more complex when you want to reach the next level of kind of like full uh kind of uh you know power of the new technology.
00:42:32.800 --> 00:42:52.719
So uh one of our customers at Adevo, uh they're doing some really great work where they've got a team that has gone full agentic, they've changed their development process as uh uh uh to um embrace AI in many different ways, requirements, gathering, transcription, story writing, implementation, all of those kind of things, and they are now going 72 times faster.
00:42:52.880 --> 00:42:59.280
So they uh have done eight years worth of work uh and they're on track to complete that in 11 months with a third of the people.
00:42:59.440 --> 00:43:01.760
So they've uh done great things there.
00:43:01.920 --> 00:43:11.599
They've uh given up on human code review, um, but each individual developer, they take an epic and then they some of them use BMAD, some of them use SpecKit, some of them vibe.
00:43:12.159 --> 00:43:25.760
Before they self-merge their PR, uh they run about seven or eight uh code reviews, different tools looking at different things, and their confidence is uh they've been doing that, that they're now confident enough that they don't need uh humans to look at it.
00:43:25.920 --> 00:43:29.119
And I think there's something interesting, I don't know how true this is.
00:43:29.199 --> 00:43:33.199
Um I mean, data is definitely like when you screw up data, your data is gone.
00:43:33.360 --> 00:43:39.119
Do we need to move to event sourcing and having a you know a replayable history of all data transformations?
00:43:39.199 --> 00:43:40.000
That's one thing.
00:43:40.239 --> 00:43:48.320
But like generally, as a trend, we've always seen that trying to prevent bad things happen by putting a check in place is the wrong answer.
00:43:48.480 --> 00:43:55.440
Like with data and databases, we went from like having transactions to prevent inconsistencies happening.
00:43:55.599 --> 00:43:57.039
That was slow, it doesn't scale.
00:43:57.119 --> 00:43:57.920
So, what was the answer?
00:43:58.079 --> 00:43:59.679
Eventual consistency, and we'll reconcile.
00:44:00.239 --> 00:44:00.960
After the fact.
00:44:01.119 --> 00:44:08.559
With the cloud native transformation, we found out that you know you can't have a thousand microservices and then integration test them all.
00:44:08.719 --> 00:44:11.119
You just have to deploy them and find out what happens.
00:44:11.280 --> 00:44:15.280
So you need great observability and you need to be able to fix forward very quickly.
00:44:15.440 --> 00:44:19.039
So again, you let the bad thing happen, but you react to it more quickly.
00:44:19.360 --> 00:44:30.719
I imagine that where we might end up with uh agentic coding is something like outcome-driven development where you've got observability of the business impact you wanted a feature to have.
00:44:30.880 --> 00:44:34.800
Like, is the application doing the things the application is supposed to do?
00:44:34.960 --> 00:44:37.519
Are users able to do what they should be able to do?
00:44:37.760 --> 00:44:51.440
And then we have software factories chucking features out, and then you look for regressions in like, okay, the number of checkouts uh on the shopping cart has has um uh has dropped and reacting and fixing forward at that point.
00:44:51.840 --> 00:44:54.639
Is that something you feel users are gonna accept?
00:44:55.039 --> 00:45:06.800
It would be interesting to see in that like do we all end up getting used to um like buttons moving locations and software being less kind of uh who was it that um coined the term?
00:45:06.880 --> 00:45:11.679
Um I think it might have been Steve Yeeg, uh might be misattributing that.
00:45:11.760 --> 00:45:23.440
Um, but of non-deterministic idempotence of this idea of you have the same spec, same product requirements, you run it through an agent twice, and you get two slightly different things, but you enable the same user behavior.
00:45:23.599 --> 00:45:32.559
Just as we've got used to waking up and finding out that you know the Apple have decided to change like how the photo album works, and our elderly parents are now very annoyed.
00:45:32.719 --> 00:45:42.800
Um, do we end up with something like that on a more daily basis, like this rapid churn of um features changing and being slightly different, but like I can still achieve the outcome?
00:45:43.119 --> 00:45:43.679
I don't know.
00:45:44.000 --> 00:45:51.039
Yeah, there was a testing Facebook, for example, like it has so many rules, so many features.
00:45:51.440 --> 00:45:53.679
Writing that kind of in a consistent test.
00:45:53.840 --> 00:46:02.400
And the way that they did it was they they added agents on that were using Facebook and they had their own rules.
00:46:02.559 --> 00:46:06.960
Like, for example, like that agent should never get a friend, like whatever.
00:46:07.199 --> 00:46:10.159
Like it should not be listed visible in anything.
00:46:10.320 --> 00:46:20.159
So they had these kind of
like flags out and by like all these kind of things, but they couldn't only make it work when they hooked them in on the production system.
00:46:20.400 --> 00:46:27.760
And that was a novel way of thinking of instead of the rigid test flow, almost like you track whatever behavior is going.
00:46:27.840 --> 00:46:35.199
If there's bad behavior, no, it's not like it's a different way of looking at observability instead of your logs, but kind of that was uh an interesting way.
00:46:35.360 --> 00:46:48.559
Like a digital twin is also something I I think will have a more future is kind of predicting things, what is gonna happen, and then the risk comes back like, okay, I I have a model, I can run it through, I can see what's happening, and kind of those things.
00:46:48.639 --> 00:46:49.280
Uh, but yeah.
00:46:49.679 --> 00:46:52.239
It's funny, I I thought about Facebook too, but from a different perspective.
00:46:52.559 --> 00:46:55.280
They have a huge infra setup for A-B testing at scale.
00:46:55.519 --> 00:47:02.719
So any developer can do whatever they want, and it'll go and get tested, A B A B testing uh facilitated again on scale.
00:47:03.039 --> 00:47:04.880
And that I think that's kind of what you were going for.
00:47:04.960 --> 00:47:06.880
How can you do it on scale and just see what wins?
00:47:07.440 --> 00:47:10.079
Um which sounds great on paper.
00:47:10.159 --> 00:47:13.599
A lot of the organizations we talked to don't have the setup for it.
00:47:13.920 --> 00:47:22.719
Uh and then where we get some organizations that come to us and figure like a few hundreds of organizations that are I think the one I talked to last week was like just shy of a thousand people.
00:47:23.440 --> 00:47:32.159
And they said that their product is just downward spiral because everybody's like a gen tech development is moving so fast, there's so much PR fatigue.
00:47:32.559 --> 00:47:37.760
The stakeholders, like the PMs and the designers and stuff doesn't go through them anymore because everything's moving so fast.
00:47:38.559 --> 00:47:44.239
And then like uh on the call was like the head of the front end infra, and he says, Yeah, I'm on calls.
00:47:44.639 --> 00:47:53.760
And I discover new features on the call that don't look anything like our other features, that don't even match our design system, that functionality-wise don't make sense to me.
00:47:54.079 --> 00:47:58.559
And I find them out with the client live, and then I asked the developer after the call, like, what is this?
00:47:58.639 --> 00:48:00.800
He's like, Oh, this got merged two weeks ago.
00:48:01.119 --> 00:48:03.280
And then I asked, did anybody look at it?
00:48:04.000 --> 00:48:05.519
Did it was there a product review?
00:48:05.760 --> 00:48:15.760
If there was like A B testing that at least it'd be guarded in some sense and not just immediately impacting all of the all of their users, and again, huge organizations, it's been around for over a decade.
00:48:16.159 --> 00:48:24.480
Uh but they have adopted agentic uh coding really quickly without amazing guardrails.
00:48:24.639 --> 00:48:25.519
And now they're kind of spiraling back.
00:48:25.599 --> 00:48:26.880
They're like, wait, wait, we gotta control it.
00:48:26.960 --> 00:48:30.559
We have a huge brand, we have users that depend on us for years.
00:48:30.800 --> 00:48:37.599
I can't move the button and make it hidden because this is a core workflow they need to work through quickly as a part of their business needs.
00:48:37.920 --> 00:48:47.840
But it's almost like uh it's the you know, it is the UI really going to be the way we you know interact with software these days?
00:48:48.159 --> 00:48:50.159
An agent will be absolutely fine with it.
00:48:50.320 --> 00:48:53.760
Uh it's us, it's again, pesky humans, which is the problem.
00:48:53.920 --> 00:48:58.320
And how much of this will actually change, and you actually think, no, I don't I don't need a UI on screen.
00:48:58.480 --> 00:49:04.480
I need to ask certain things, or I just need my agent to go ahead and interact in in a specific way.
00:49:04.639 --> 00:49:05.599
Um so I don't know.
00:49:05.679 --> 00:49:13.920
It's it it if I I wonder if in years buttons and things like that will just be replaced with completely different ways of us interacting with apps through.
00:49:14.400 --> 00:49:18.960
Yeah, but we'll still be a visual-oriented creature as well, right?
00:49:19.039 --> 00:49:38.800
You we can't just say we're just typing in the words, like you know that uh there was a talk here as well from TL Draw, kind of interesting that like you know, having a your agent giving it a whiteboard, because sometimes you can express things easily in the whiteboard and like typing and and that kind of fluidity is also, I think, in the in the UX.
00:49:38.880 --> 00:49:40.559
Like sometimes it is gonna be.
00:49:40.719 --> 00:49:42.880
I I just want to ask a question, you reply back to me.
00:49:43.039 --> 00:49:45.280
Sometimes just show me, like explain it to me.
00:49:45.519 --> 00:49:48.400
But it might be interactive and kind of going through that.
00:49:48.559 --> 00:50:01.039
I think that's also the pendulum, which was funny, like IDE, CLI, and now we're back at orchestrator review tools because we need something, like we need to see what's happening, and we can't just like do that.
00:50:01.280 --> 00:50:07.920
But the agent might crack along in their own communication and find the best way of kind of dealing internally.
00:50:08.239 --> 00:50:15.119
But as long as we need to make a call uh shots on kind of those uh kind of risks, we need to be informed.
00:50:15.199 --> 00:50:18.159
And somebody explains it to me like what are we doing here?
00:50:18.320 --> 00:50:20.960
Uh and that's a challenge if we don't do it ourselves anymore.
00:50:21.039 --> 00:50:35.519
So we've talked a lot about good patterns, about ways in which people should should change the way they develop software, um, and to
essentially directing people in groups as to how they should change.
00:50:36.079 --> 00:50:47.519
But what is it that organizations, uh, enterprises are doing today as a part of their migration, part of their adoption, that is actually just a mistake.
00:50:47.599 --> 00:50:52.639
It's wrong, it's a it's a it's been proven that you know oh dragons live here.
00:50:53.199 --> 00:50:55.760
What should they what should they not do that they're doing today?
00:50:57.920 --> 00:51:10.000
I think I'd I'd give them the advice is that there's the still the tendency of like if we're doing a content constant conversation with our agents, we're in control.
00:51:10.719 --> 00:51:16.239
I think there needs to be um for them to get into the next mile, they have to let go.
00:51:16.639 --> 00:51:21.119
But they it's it's almost like you have to mentor the other one instead of doing the work.
00:51:21.199 --> 00:51:25.199
And that's kind of the mental challenge that I think a lot right have right now.
00:51:25.280 --> 00:51:27.679
Like um stop doing the work yourself.
00:51:27.920 --> 00:51:31.519
And it's it's really weird if that's what you've been doing for so many years.
00:51:32.079 --> 00:51:39.760
And what a lot of developers actually hold uh important identity cells because it's what make it's what they feel sometimes actually makes them the expert, right?
00:51:39.920 --> 00:51:40.719
Yeah, correct.
00:51:41.199 --> 00:51:41.760
It's interesting.
00:51:41.840 --> 00:51:43.599
So I have a few different answers to this.
00:51:44.559 --> 00:51:48.239
Uh the first one is you want you want to control your AI spend.
00:51:48.400 --> 00:51:53.360
People are are releasing crazy budgets and everybody's trying to token max every single developer.
00:51:53.519 --> 00:52:07.360
Uh and if you I think the right way is give the developers the task that interests them, you will see they're way more engaged with the agent in the session where they're building it, versus give them the task that the developer's not interested, and then they will offsource all that mentor load to the agent.
00:52:07.599 --> 00:52:12.480
Uh one, you get poor quality PRs, but two, you get way bigger token spent.
00:52:12.719 --> 00:52:21.599
Like imagine that something's out, the the disinterested developer will just say agent test it instead of testing themselves and seeing exactly what's wrong and saying, oh, this is the issue, let's fix it.
00:52:21.760 --> 00:52:29.280
They will let the agent do seven sessions of QA and the token count for the session goes crazy because the developer is not interested.
00:52:29.840 --> 00:52:31.440
And then, so that's one part.
00:52:31.599 --> 00:52:38.159
Like, yes, people can move faster, giving them tasks they don't care about is a great way to spend your AI budget really quickly.
00:52:38.480 --> 00:52:38.880
That's one.
00:52:39.039 --> 00:52:44.239
The other one that we've seen a lot of organizations do is they want to jump in headfirst to AI native.
00:52:44.559 --> 00:52:47.920
Uh and they do that by giving cloud code to everybody.
00:52:48.400 --> 00:52:51.039
And the cloud code has code in the name.
00:52:51.280 --> 00:52:52.320
It is great for developers.
00:52:52.480 --> 00:52:53.199
I love it.
00:52:53.519 --> 00:52:58.880
It is most definitely not good uh for non-developers, and if you give them that, you will get PRs.
00:52:59.599 --> 00:53:09.360
One, they don't know what they're doing, they don't know how to set up an environment, and they're gonna drive your dev team crazy asking questions, even if they feel comfortable enough asking questions, or they're just not doing any meaningful work.
00:53:09.679 --> 00:53:13.679
But if they do open PRs, we see like garbage in, garbage out.
00:53:14.000 --> 00:53:24.639
Uh PRs that the VP we're on calls with like CTO and VP product, and the VP product shows off to us saying, Hey, I opened a PR yesterday, and then the CTO sitting next to him says, Yes, it was crap.
00:53:25.360 --> 00:53:28.800
Um that also a great way to not go about it.
00:53:29.280 --> 00:53:34.559
Uh so when you do enable them, you want to get the tool that is really optimized for that user.
00:53:35.280 --> 00:53:43.360
Again, we optimize for PMs and non-developers, but giving the people the tool that's not meant for them is not the answer.
00:53:43.519 --> 00:53:48.639
Uh again, it you use your your AI budget really quickly, and you get frustrated people all around.
00:53:48.719 --> 00:54:02.800
The PR fatigue is there on the developer side, and PMs that open really poor PRs are increasing the PR fatigue, they're getting more frustrated themselves, and you get Uber scenarios where everybody's just disappointed.
00:54:04.320 --> 00:54:23.119
I think uh in terms of what people should not do or stop doing, like the whole uh piecemeal approach to last year, 2025, was very much the year of CTOs going, well, I let people use their own tools and whatever they're comfortable with, like that is going to end up being inconsistent and you're not gonna get the organizational learning uh that you need.
00:54:23.199 --> 00:54:27.440
So affirm mandates that like the AI bus is leaving, please get on board.
00:54:27.599 --> 00:54:36.960
Um, that's been shown in research by multitudes, I think, to uh be uh a strong uh indicator of uh success of a genetic coding adoption.
00:54:37.360 --> 00:54:46.159
Um the the counterpoint to the um token maxing is also don't go to the other extreme and set a really low limit.
00:54:46.320 --> 00:54:54.719
Um, you know, I've I've seen places where there's a hundred euros a month uh tokens uh limit, and like 100 euros you can't really do very much.
00:54:54.800 --> 00:55:02.079
So it ends up adding cognitive load of people like, oh, should I should I use Cursor for this or should I is this the most important thing I'm gonna do this month?
00:55:02.239 --> 00:55:07.840
Um so having something in between um would uh would also be sensible.
00:55:08.079 --> 00:55:16.800
And then maybe rethinking what a test is, but you know, to invoke Beyonce, if you liked it, then you should have put a test on it.
00:55:17.039 --> 00:55:23.519
Um if you care about a thing, then you need to have something in place to assert that.
00:55:23.679 --> 00:55:39.280
Whether that's functional tests, whether it's a uh a sweep of an agent with a particular lens looking at a concern, whether it is connecting up via MCP or agent to observability so it can figure out the thing that it just uh built and got deployed is uh now breaking.
00:55:39.440 --> 00:55:43.840
Um, making sure that there are those feedback loops in place.
00:55:44.320 --> 00:55:53.679
If you just have an agent on its own and you expect it to get everything right first time without being able to even perceive the mistakes it might make, then you're gonna get bad results.
00:55:53.920 --> 00:56:07.440
So um making sure that you think about what you value in your code, um, that it's architecturally uh you know uh sound and um the code quality is high and the test coverage is good.
00:56:07.599 --> 00:56:13.840
You need to work out what you value and then make sure that there is something in place to prove that that is there.
00:56:14.000 --> 00:56:23.519
So it's not quite testing, but it's like thinking about the the non-functional aspects of your code as if they were tests, if that makes sense.
00:56:24.000 --> 00:56:25.039
Okay, expand slightly.
00:56:25.440 --> 00:56:26.320
I love that point.
00:56:26.559 --> 00:56:39.039
Uh I think something that's really important is also the temporal mindset and the context hinting mindset, which means every test needs to stand the test of time, needs to be evergreen, but you also need to think of it as an opportunity for a context hint.
00:56:39.360 --> 00:56:50.480
Meaning when the test something fails, the test has enough context in whatever the assertion is, uh the comment, the error response, whatever it may be, so that the agent that gets it will know how to act.
00:56:50.719 --> 00:56:53.440
So lazy developers just say, cool, I covered it with tests.
00:56:53.519 --> 00:57:00.400
No, like you need to cover it with tests, and you need the test to have the context hints that the future agent that will see them will know what to do with it.
00:57:00.559 --> 00:57:04.559
Like that's been really impactful in getting the agents to actually manage in sophisticated code bases.
00:57:04.800 --> 00:57:10.800
Yeah, and it's exactly the same as we were uh all saying earlier of like that's always been good development practice.
00:57:10.880 --> 00:57:16.559
Like it's nothing worse than as a developer, you run the tests and you just get like assert true failed, and you're like, great.
00:57:16.880 --> 00:57:18.159
Or 1317.
00:57:18.239 --> 00:57:20.800
Yeah, well, what do I do with this now?
00:57:20.960 --> 00:57:22.559
So yeah, it's uh important stuff.
00:57:22.800 --> 00:57:29.519
And thank you for quoting Beyoncé, because uh I've been trying to work out how I increase that in this podcast, but but finally someone has.
00:57:29.599 --> 00:57:30.639
So thank you very much.
00:57:30.800 --> 00:57:37.119
Um, well, we've talked about a lot, and it's been super, super valuable to have you all kind of like discussing on these points.
00:57:37.199 --> 00:57:39.599
So I appreciate appreciate all your input on that.
00:57:39.760 --> 00:57:43.519
Um, thank you very much for joining us at AI Native DevCon.
00:57:43.679 --> 00:57:49.199
Why don't you very briefly uh talk a little bit about some of the sessions that you're each giving?
00:57:49.360 --> 00:57:50.400
Patrick, why don't you start?
00:57:50.639 --> 00:57:50.880
Yeah.
00:57:51.039 --> 00:57:58.159
Um I'm talking about the different layers from up to a dev to a team to uh a V a platform team and a VP.
00:57:58.320 --> 00:58:02.079
What's the mindset that you need to have to go into this new agentic world?
00:58:02.239 --> 00:58:08.400
I'm not talking about generic transformation, like do a hackathon and so on, but what is really like how do you embrace the new thing?
00:58:08.559 --> 00:58:09.360
So that's my talk.
00:58:09.679 --> 00:58:10.320
Awesome.
00:58:10.639 --> 00:58:24.239
Uh so my talk is uh about what it's like to become more of an AI native organization, where your PMs, your non-developers are actually opening PRs, and how really the metric you should be copying, actually should be tracking is the merge rate.
00:58:24.400 --> 00:58:27.360
So the the PRs that they open, don't say how many PRs do you open.
00:58:27.599 --> 00:58:28.880
Look at what the merge rate is.
00:58:29.039 --> 00:58:33.039
That's how you will know are they opening quality stuff or are they just increasing PR for T.
00:58:33.280 --> 00:58:33.679
Yeah.
00:58:34.000 --> 00:58:43.199
Uh I was uh co-presenting with Tomas Mai uh from Adevo, and we were talking about the story of how we upskilled 120 of their developers in agentic coding.
00:58:43.280 --> 00:58:48.559
And the most important thing about that talk is you get to see me in a day glow fluorescent uh tracksuit.
00:58:48.800 --> 00:58:50.800
So you should people should definitely check that.
00:58:51.280 --> 00:58:55.679
Beyonce end this, really is there a beyond is there a Beyoncé uh quote in that one?
00:58:55.920 --> 00:58:57.599
I don't think there is Shakira, maybe?
00:58:58.480 --> 00:58:58.960
Sadly not.
00:58:59.199 --> 00:58:59.760
Alright, alright.
00:58:59.920 --> 00:59:08.000
Um so uh why don't we uh if if well all of these talks are recorded, so you're very, very welcome to go and we'll we'll add some we'll add some links there in the show notes.
00:59:08.159 --> 00:59:15.119
If I or my our listeners can only go and watch one talk, which one would you agree that they should No, no, let's not let's not do that.
00:59:15.199 --> 00:59:16.960
Uh all of them, all of them.
00:59:17.119 --> 00:59:18.480
Um thank you very, very much.
00:59:18.639 --> 00:59:21.039
Really appreciate you being here and uh sharing the time with us.
00:59:21.119 --> 00:59:21.440
Thank you very much.
00:59:21.760 --> 00:59:22.320
Thanks for watching.
00:59:22.400 --> 00:59:22.559
Thank you.
00:59:22.800 --> 00:59:23.679
I hope you enjoyed that.
00:59:23.760 --> 00:59:25.760
That was really great from my point of view.
00:59:25.840 --> 00:59:28.159
Uh would love uh to hear any feedback on that.
00:59:28.239 --> 00:59:28.880
Please let us know.
00:59:29.039 --> 00:59:32.239
Podcast at Tesla.io and uh tune in to the next episode.
00:59:32.400 --> 00:59:33.199
Bye for now.
00:59:33.519 --> 00:59:37.360
Hope you had as much fun listening to this episode as I had recording it.
00:59:37.440 --> 00:59:44.480
And if you want to hear more from the guests featured in this episode, we have their full talks linked in the description below.
00:59:44.639 --> 00:59:45.840
So that's it for this episode.
00:59:46.079 --> 00:59:47.039
See you next time.
00:59:47.599 --> 00:59:52.400
The AI Native Dev is brought to you by Tesla, the package manager for Skills and Context.
00:59:52.480 --> 00:59:55.519
Your hosts are Guy Pajani and me, Simon Maple.
00:59:55.679 --> 00:59:57.119
Our producer is Tom Dowler.
00:59:57.440 --> 01:00:00.639
The AI Native Dev is not just a podcast, it's a community.
01:00:00.800 --> 01:00:04.079
And we host a monthly meetup at the Tesla offices in Temple Land.
01:00:04.559 --> 01:00:08.639
Visit Tesla.io for community to learn more of iOS.