Anne Currie: Thank you very much for listening to Asynchronous and Unreliable podcast. If you enjoyed the show and you want us to create more content, please do show your support by hitting the subscribe button below. It really does make a difference. Thank you very much. to a asynchronous and unreliable podcast. I am your host, Anne Currie author of many books, ⁓ lot of which are about AI. And today we're going to do something a little bit different ⁓ we're going to talk about ⁓ last podcast Martin Davidson's ⁓ talk about his experimentation with getting AI write mission critical software. Now, the obvious thing After hearing that and listening to that, it's a fantastic episode. You don't have to have listened to it to listen to this one. But I would imagine I would strongly recommend that you do listen to it at some point because it is really it feels like it's putting forward an idea that is essentially game changing, which is that AI could potentially write mission critical software. Now, after hearing Martin's talk, obviously, my next step was it could not be anything other than to talk to somebody who used to manage teams and organizations producing mission critical software. Because I want to get ⁓ feedback of Jon Berger, who's been on ⁓ podcast before. ⁓ want to hear your thoughts after listening to Martin. What are the questions that you got in your mind? What are the possibilities that it's unlocked for you? This is all about noodling out some thoughts. ⁓ welcome again, Jon Don't worry too much about this being like a fully formed thing. We are at this point both going, after what we just heard, which is quite surprising and quite interesting and quite unlocks a load of possibilities that we hadn't thought would be available for years to come, if ever. Or maybe not. But maybe you did think it was coming. So I will hand over to you. Initial thoughts.
Jon Berger: Okay. I mean, there are a lot of thoughts. was a great podcast with Martin. He's a smart guy and I have a hundred sort of follow-up questions on all the different bits that you talked about, which was really interesting. I I think in terms of when you say this was a surprise to us and we maybe never thought this was coming, you're thinking from like us of five years ago, not from us of immediately before the podcast, just to be clear. We've been following enough of what's going on and talking to enough other people like Martin who are towards the cutting edge of ⁓ using AI for writing software. Not a lot of what he said in the podcast was a surprise to us, but it certainly would have been a big surprise to us of not that long ago. And I think it will be interesting to a lot of people.
Anne Currie: And actually, I think it's worth saying that you it wasn't as much of surprise to you, I think, as it was to me, because you were a contributor to Building Green Software, the O'Reilly book that me and Sarah and Sara wrote. And it came out what, three years ago now? Two years ago now? ⁓ Two years, two years ago. Time flies. There's a, but we wrote it, wrote a lot of it, wrote the code, the kind of feature of code efficiency chapter more like three years ago. And you were a big contributor to that. And you kept saying to me, I put in a lot about maybe A.I. is going to completely revolutionize efficient code because you kept saying it will because this is the area that you were in and you're already starting to see it before I think the majority of us were. So I took your word for it and put in a whole load of stuff in the in the O'Reilly book about things are going to change here, everybody. I'm reliably informed. So obviously this wasn't a total shock to you.
Jon Berger: No, think it was obviously the direction of travel. It's still the direction of travel, right? mean, what we see, the capabilities we see today will change between us recording this and you publishing it, even if that's only 24 hours, right? It's a really fast moving space. so, one mistake people are making at the moment is looking at... today's capabilities and assuming that that's the world that they're trying to adjust to. ⁓ I think Wayne Gretzsky would be trying to skate to where the puck is going to be, not to where it is today. ⁓ And that's hard to say exactly where it's going to end up. That it seems like with the current direction of travel, like there is way more to come from AI. ⁓ would bet that there will be a point in the future where AI just writes all the code and humans just don't look at it. ⁓ just seems like it's pretty inevitable from where we are now. ⁓ That is not where we are now, just to be clear. I don't believe we've got there yet. ⁓ I do believe that that is definitely coming. Is that a week away, a year away, five years away, 10 years away, 20 years away? Yeah, one of those definitely. Definitely within 20. Likely within 10 and plausibly much closer than that. So I think that the overriding impression I have is guess really to paraphrase something you've been saying for ⁓ a while now, ⁓ good at change. The modern software world is ⁓ whether you like it or not. ⁓ to like it, try to embrace it, get good at it. And maybe ⁓ should explain that sort of get good at change statement from where you're coming from.
Anne Currie: it's interesting that you say that. ⁓ 10 years ago, I wrote a book, which is actually sitting behind you, Jon, there, ⁓ the cloud attitude, in which I went out and I ⁓ interviewed whole load of enterprises all over, actually, to be honest, mostly all over Britain, because they they were easy to get to. ⁓ about who were doing cutting edge stuff with cloud, who were doing cutting edge stuff with cloud native, what I discovered was that ⁓ actually, ⁓ generally speaking, progress was for most businesses still really But in businesses, where the work was ⁓ going fast and they had adopted cloud native. They were going full fast flow ⁓ using the kind of highly iterative techniques that were pioneered by Toyota, the Toyota production system. And they were going ⁓ fast and fast feedback with using feedback loops to be deploying changes, tiny changes, a very large number of times per day. And it was that kind of trial and error iterative process that was meaning that they could get their heads around using the new tools in the cloud because there were so many new tools. And it was more about that attitude than it was about any individual tool. They were all using different tools. There was no technology that everybody was aligned on to be successful. The only thing that they were aligned on to be successful was the ability to make change and to make change fast by taking small steps.
Jon Berger: Yeah, because they all had different problems to solve. So there wasn't just a just follow this recipe and you'll be fine. And there was a lot of things that were fragile or new or still being developed and they needed to they needed to go through the failure processes to learn what worked for them or what didn't. And I think the same sort of logic applies now for these sorts of changes. So, yeah, get get good at change, embrace it. because Change is coming for everyone.
Anne Currie: It certainly is. And to certain extent, it's interesting. We've discussed this many, many times, how radically our business has changed. The tech industry has changed throughout the 30 years that we've been in it. So we've been both, basically we both started up work on at the same time, within the same week, within the same company, more than 30 years ago. now, 32 years ago. And during that time, everything has changed again and again and again. And it is about to change again, at least as well. And the big change that that Martin Davidson described in his podcast, which is now live, you can you can look at it. It's potentially more impactful than everything else we've seen. It's like everything else we've seen all rolled up into one and going in one release, isn't it?
Jon Berger: Yeah, I think that's true. I think, sort of stepping back, the big picture sort of thoughts and questions I had coming out from your conversation with Martin was, first of all, Martin is someone I trust, right? We've known him for 30 years. He's worked on complex software. software which has network effects of where performance like really, really high scale performance matters, where scalability matters, where reliability matters. And so this is not someone who's just written an app that shows you cat videos. Not that that's necessarily trivial, but it's not.
Anne Currie: Don't let that's anything wrong with that
Jon Berger: may or may not be something wrong with that, but that's not that we're not making a judgment. The point is this is someone who does understand what complex looks like in the software industry saying, I don't need to look at this code anymore. The AI just does it with me. That's that is interesting. And the other thing that sort of goes along with is we've been We've we've we've been plugged into a lot of people who've been talking about this for a long time. And there's there's a lot of noise out there from mostly from people saying, AI it's rubbish. It can't do this job. And, know, they don't say at the moment, they just say it can't do it. They sort of they leave the implication to you. If I want to learn to swim, I don't go to all the people who can't swim and say, how do you swim? to ⁓ to the people who can swim and say, how do you swim? Right. So the people that say, ⁓ can't do this job at the moment. ⁓ sort of irrelevant. ⁓ relevant people, the interesting people, the people you can learn from a people are saying, this is what I've done and it worked for me. That's not the same as it will work. identically for you, but there's something there if someone's found something that can work. And Martin and many others have got something that can work here. Where there's a meaningful step up compared ⁓ where we ⁓ not that long ago.
Anne Currie: Yeah, he said that himself, and he thinks have changed quite radically. the interesting thing here, and you did get mentioned in the couple of times actually in the Martin podcast, because around kind of stuff that he's been building to try out, ⁓ the of stuff that you used to be responsible for delivering ⁓ with gigantic teams of super highly ⁓ trained engineers like SIP stacks ⁓ How does that make you feel?
Jon Berger: It's brilliant, isn't it? I mean, just to be clear, I think when you say gigantic teams, mean in the modern word of gigantic meaning there was more than one person on the team. It wasn't just one person and some AI. I've always worked in what for most of the software engineering industry would be considered sort of smallish teams, sort of five to eight, which is a nice size for minimizing ⁓ pain in terms of team churn and in terms of communications overhead and just sort of like the coordination overhead of large teams. But that may well seem gigantic for someone like Martin, whose team is him and as many AIs as his credit card will allow. ⁓ I think it's fascinating and it's a major proof point because writing a SIP stack is non-trivial. It's not the hardest thing in the world, but it's more complex than probably most software, I would say. it does have performance requirements and it does have... But on the other hand, it's pretty testable. compared to some things, you know, and it is reasonably well specified, which makes it a good starting point.
Anne Currie: So I will bridge this thing. SIP stack, so SIP is a networking protocol. a lot of calls and ⁓ calls over IP, voice over IP runs on top of a protocol called SIP. that was kind of because you were ⁓ managing and leading teams of engineers that delivered networking software. a SIP stack was a kind of a fairly common piece of networking software to be developed at a period of time everyone was developing their SIP stack and that was a period of time in which we were quite active in, ⁓ well, very active in the tech industry and in the networking industry. Is that correct?
Jon Berger: Yeah, mean, SIP is a protocol used between entities that wants to communicate ⁓ to essentially negotiate how that communication is going to happen. So it's what we'd call the control plane. It's about how the call is going to work rather than the data plane, which is the thing that actually supports the call itself. Although Martin said he'd also done that as well. So there you go. Why not? Because you had a spare five minutes. What you didn't yet talk about with Martin and the obvious question is, so if one interpretation of what Martin said is software engineering is solved, great! ⁓ mean, there is something we should come back to there, which is that his new his new flavor of software engineering, actually. But software engineering is solved. Great. ⁓ Well, OK, a lot of people be going, well, wait a minute, but what about? And and there are a lot of what abouts, but to me, the big ones are sort of the maintenance and operability side. Like, OK, I as a one man band can create some software that works for me straight away. And it sort of basically works. And I am both the engineer and the customer. Maybe that's the future of all software. All software will have exactly one user, and the user is the person that says to their friendly AI, please write me some software that does whatever I need it to do. And there is no scalability and limited performance requirements for most software other than. the AI itself and whatever networking is required to allow you to talk to it, because all software is bespoke. That's one possible world we might end up in. ⁓ But I doubt it. You know, I can't really ever see my mom doing that. ⁓ So there will be still be a software industry in which some software is is sold to other people. Some level of service maybe is sold to other people. And therefore there is someone who's receiving that cash who's on the hook to ensure that that service works. And I think that's one of these get good at change areas, which is at some point in the next 10 years, almost all of that work maybe is done by an AI for you or a series of AI agents. That isn't true today. No one, think, is so convinced about the correctness of the AI generated software. that they've handed over the entire operations side to an AI to just go, yeah, please run my business for me. Let me know in a year's time when it's time to cash check. That's not yet. So there's a whole series of business processes and team interactions here that needs to evolve along with what the technology can do. Because they don't evolve along with what the technology can do, then anyone who is evolving alongside it is going to eat your lunch. They're going to go faster than you are. But if you if you evolve too quickly, then maybe there's a there's a challenge there. But that's where getting good at change is going to happen.
Anne Currie: ⁓ Yeah, because I mean, it seems so long ago, but it's like two years ago, everybody reckoned the new skill would prompt engineering. And now we know that the best way to produce a good prompt is just to ask to do a basic prompt into chatgpt or whatever to give you a good prompt for what you're attempting to do. And it's so much better and it'll give you back a really good prompt.
Jon Berger: Thank
Anne Currie: prompts engineering is dead. The AIs just much better at prompts engineering than we ⁓ are.
Jon Berger: Well, they're both better at it in that sense. You can ask it to write prompt, but also just the thinking models are better at just doing that as they go. can give them something simple and you can see them creating their own prompt as they're going.
Anne Currie: Yeah, it is quite astonishing. ⁓ Yeah, the time when which you write, you carefully craft your own prompt is very much at an end. For most of them, you should be asking that the chatbot will help you do that, we'll give you a first pass.
Jon Berger: Yeah. Carefully crafting your own prompt is still a good idea alongside AI as opposed to doing something on your own. It's just one of that. That is one of a whole class of things where you probably want to try using AI alongside to help you.
Anne Currie: ⁓ I one of the interesting things in Martin's, ⁓ when Martin was describing how he learned to ⁓ program differently in order to Claude and Codex, was when he said, you've got to define what good looks like upfront. The irony is that all my books and everything had been about how progress has happened through trial and error and iteration. And he said, well, the problem with that is that you can't do trial and error really, because it's just going so much faster than you. You need to say, well, I think I want something where good will look like this. Go for it. And then at the end, go, did I? Well, let's refine that and go for it again. Which could say a different form of trial and error, but it means you have to think in much bigger chunks.
Jon Berger: trying to narrow a different scale. Yeah, you've still got to have a sense of what you're trying to achieve, which as Martin said, was always good management anyway. It's like, ⁓ and build me something and then I'll tell you whether it was good or not. ⁓ not a good place to be. ⁓ it's still not a good place to be. And now, and now AI can call you out on that without anyone feeling bad.
Anne Currie: you You Yeah. Yeah. There's an excellent Mitchell and Webb look sketch. Not the one about are we the baddies, which obviously everybody's seen millions of times. There's an, would say there's an even better one about ⁓ a creative trying to get a good brief from an exec who basically cannot brief at all and is trying to basically constantly going, ⁓ time you produce something, it's like that, but not like that, obviously. ⁓ like that, but not like that. their It's a terrible way of getting stuff out of creative humans. And Martin's point is, it's a terrible way of getting stuff out of AIs as well. ⁓ It's your management skills that you need to be sharpening up
Jon Berger: Yeah, it is. But I wouldn't want people to think, well, you've got to go away and spec the whole thing in advance before you can achieve anything, because AI is really good at the spec inside as well. And and in the helping you to refine your thoughts. Now, even if it's just a rubber duck where you say you are, want something like this and it and just by hearing yourself say it, you go, ⁓ wait a minute, that wasn't very clear or.
Anne Currie: You
Jon Berger: actually, no, don't want that or I can add some more or whatever, but it's, it's, you can use AI to do a lot of the legwork of that spec. And in fact, it will result in much better quality output. If you say, well, this is the sort of thing I want. What, what, what, what questions do you have? Can you write yourself a spec and review it and ask me about any questions you have and And you you work with the the that first level agent on the spec before you then move on to the actually now go and build the thing.
Anne Currie: Yes, it's not the throw over the wall at the AI. It was, ⁓ I think, one the most interesting things ⁓ on Martin's when was describing his new SDLC, his new AI software development lifecycle was, ⁓ kept saying we the time when he meant me and the other, ⁓ the chat. Well, was a confused by that because I thought, who's the other people? no, it's Claudian Codex. ⁓ I thought it was quite interesting and it quite amused me that he said, ⁓ ⁓ one wants, everybody likes to work in a team. No one wants to get rid of all ⁓ humans in their team. But it was also at the same time, quite clear that he worked on his own, although he's a very sociable person, he's a lovely, lovely guy. ⁓ He worked on his own quite happily with the chatbots and he referred to them as weak. ⁓
Jon Berger: And that's.
Anne Currie: Yeah.
Jon Berger: Yeah, I mean, it's a very positive sign, isn't it? That he uses the word we. One of the indicators of ⁓ a ⁓ less than great, let's say, software engineering organization is when the different teams do just throw stuff over the wall to each other, when product management just throw something over the wall to the engineering side, or if you have a separate test team. things get thrown over or operations. It's like, and you want all those people to be using we building things together because they will be more successful. But yeah, that was interesting. mean, the other part of that was I think there's been there's been angst in the software industry for. almost as long as I can remember that people that went into the business to write code, that was that was how they saw themselves. They wanted to write code. They enjoyed the writing of the code. Often they didn't really enjoy a lot of the other bits. They didn't really like writing specs. They didn't really like testing it. You know, all of those all of those other things. ⁓ But they they saw themselves coders and and ⁓ is not the valuable bit. It's always been an interestingly difficult bit. And there's always been an awful lot more people in the world that couldn't do it than that could, which is why we've made money out of that in the past. But software engineering as as Martin now views it is, well, my goal is to create something that does something useful. I don't really care who writes the code or who writes the spec or who does the testing or in fact.
Anne Currie: you
Jon Berger: Actually, I sort of do care. I'd really like my AI to do it all for me because that's a million times faster or more. And, you know, I don't have to do all the bits that I don't want to do. And he views it as we, as you say, we created this software. He's not even seen the code. And he feels really happy about that. But that's a great thing. He feels happy about that. Isn't that brilliant?
Anne Currie: So it's an interesting one to say, because he pointed out very correctly that we don't see the assembly language, we used to in our earlier careers, we were very familiar with the assembly language. ⁓ I haven't looked at a piece of assembly language in some decades. ⁓
Jon Berger: No. And how many times did you ever look at a voltage on a chip? Never. Never. Never did that. So I never really knew what was going on in this thing that people told me it was a computer, but might just be magic.
Anne Currie: That's true. Yeah, but I never referred to me in the compiler as we.
Jon Berger: You're just not a team player and that's the problem.
Anne Currie: Maybe it's because I don't build in Rust and the Rust compiler is much more sophisticated. Maybe the Rust compiler would be my pal and I would think about what the Rust compiler was doing while I wasn't around in the room. But the C compiler was never, we were never on friendly terms, we were never on we terms.
Jon Berger: Yes.
Anne Currie: Now, but so we've done half an hour and that was really interesting. And we are going to ⁓ have more conversations, I think, about Martin, because I specifically want to I want to start diving in on some of the things that I thought he said that were very interesting about the kind of although this might happen with you or it might happen with with somebody else who's more expert than either of us two on on this. I was really interested in his comments on on testing and how it expanded. the realms of what was a plausible testing like ⁓ multivariate testing for the edge cases on conditionals if tests, It was interesting that he was saying, well, you just wouldn't do that except on air traffic control systems. But now you can. That's going to change everything. we might talk again a bit about that. But there's so much here to talk about. I'll see if I can get a couple of Rust experts in and a couple of testing experts in. And we can just untangle this a little bit more. But thank you very much for this chat. And we are. As I say, you're very usefully cached. locally, so we will definitely have more discussions on this podcast about what we've just heard and what we're learning. But thank you very much, Jon. And see you again. See everybody see all the listeners and watch us to this podcast again soon on asynchronous and unreliable podcast.
Jon Berger: Thank you. That was fun.
Jon Berger: Okay. I mean, there are a lot of thoughts. was a great podcast with Martin. He's a smart guy and I have a hundred sort of follow-up questions on all the different bits that you talked about, which was really interesting. I I think in terms of when you say this was a surprise to us and we maybe never thought this was coming, you're thinking from like us of five years ago, not from us of immediately before the podcast, just to be clear. We've been following enough of what's going on and talking to enough other people like Martin who are towards the cutting edge of ⁓ using AI for writing software. Not a lot of what he said in the podcast was a surprise to us, but it certainly would have been a big surprise to us of not that long ago. And I think it will be interesting to a lot of people.
Anne Currie: And actually, I think it's worth saying that you it wasn't as much of surprise to you, I think, as it was to me, because you were a contributor to Building Green Software, the O'Reilly book that me and Sarah and Sara wrote. And it came out what, three years ago now? Two years ago now? ⁓ Two years, two years ago. Time flies. There's a, but we wrote it, wrote a lot of it, wrote the code, the kind of feature of code efficiency chapter more like three years ago. And you were a big contributor to that. And you kept saying to me, I put in a lot about maybe A.I. is going to completely revolutionize efficient code because you kept saying it will because this is the area that you were in and you're already starting to see it before I think the majority of us were. So I took your word for it and put in a whole load of stuff in the in the O'Reilly book about things are going to change here, everybody. I'm reliably informed. So obviously this wasn't a total shock to you.
Jon Berger: No, think it was obviously the direction of travel. It's still the direction of travel, right? mean, what we see, the capabilities we see today will change between us recording this and you publishing it, even if that's only 24 hours, right? It's a really fast moving space. so, one mistake people are making at the moment is looking at... today's capabilities and assuming that that's the world that they're trying to adjust to. ⁓ I think Wayne Gretzsky would be trying to skate to where the puck is going to be, not to where it is today. ⁓ And that's hard to say exactly where it's going to end up. That it seems like with the current direction of travel, like there is way more to come from AI. ⁓ would bet that there will be a point in the future where AI just writes all the code and humans just don't look at it. ⁓ just seems like it's pretty inevitable from where we are now. ⁓ That is not where we are now, just to be clear. I don't believe we've got there yet. ⁓ I do believe that that is definitely coming. Is that a week away, a year away, five years away, 10 years away, 20 years away? Yeah, one of those definitely. Definitely within 20. Likely within 10 and plausibly much closer than that. So I think that the overriding impression I have is guess really to paraphrase something you've been saying for ⁓ a while now, ⁓ good at change. The modern software world is ⁓ whether you like it or not. ⁓ to like it, try to embrace it, get good at it. And maybe ⁓ should explain that sort of get good at change statement from where you're coming from.
Anne Currie: it's interesting that you say that. ⁓ 10 years ago, I wrote a book, which is actually sitting behind you, Jon, there, ⁓ the cloud attitude, in which I went out and I ⁓ interviewed whole load of enterprises all over, actually, to be honest, mostly all over Britain, because they they were easy to get to. ⁓ about who were doing cutting edge stuff with cloud, who were doing cutting edge stuff with cloud native, what I discovered was that ⁓ actually, ⁓ generally speaking, progress was for most businesses still really But in businesses, where the work was ⁓ going fast and they had adopted cloud native. They were going full fast flow ⁓ using the kind of highly iterative techniques that were pioneered by Toyota, the Toyota production system. And they were going ⁓ fast and fast feedback with using feedback loops to be deploying changes, tiny changes, a very large number of times per day. And it was that kind of trial and error iterative process that was meaning that they could get their heads around using the new tools in the cloud because there were so many new tools. And it was more about that attitude than it was about any individual tool. They were all using different tools. There was no technology that everybody was aligned on to be successful. The only thing that they were aligned on to be successful was the ability to make change and to make change fast by taking small steps.
Jon Berger: Yeah, because they all had different problems to solve. So there wasn't just a just follow this recipe and you'll be fine. And there was a lot of things that were fragile or new or still being developed and they needed to they needed to go through the failure processes to learn what worked for them or what didn't. And I think the same sort of logic applies now for these sorts of changes. So, yeah, get get good at change, embrace it. because Change is coming for everyone.
Anne Currie: It certainly is. And to certain extent, it's interesting. We've discussed this many, many times, how radically our business has changed. The tech industry has changed throughout the 30 years that we've been in it. So we've been both, basically we both started up work on at the same time, within the same week, within the same company, more than 30 years ago. now, 32 years ago. And during that time, everything has changed again and again and again. And it is about to change again, at least as well. And the big change that that Martin Davidson described in his podcast, which is now live, you can you can look at it. It's potentially more impactful than everything else we've seen. It's like everything else we've seen all rolled up into one and going in one release, isn't it?
Jon Berger: Yeah, I think that's true. I think, sort of stepping back, the big picture sort of thoughts and questions I had coming out from your conversation with Martin was, first of all, Martin is someone I trust, right? We've known him for 30 years. He's worked on complex software. software which has network effects of where performance like really, really high scale performance matters, where scalability matters, where reliability matters. And so this is not someone who's just written an app that shows you cat videos. Not that that's necessarily trivial, but it's not.
Anne Currie: Don't let that's anything wrong with that
Jon Berger: may or may not be something wrong with that, but that's not that we're not making a judgment. The point is this is someone who does understand what complex looks like in the software industry saying, I don't need to look at this code anymore. The AI just does it with me. That's that is interesting. And the other thing that sort of goes along with is we've been We've we've we've been plugged into a lot of people who've been talking about this for a long time. And there's there's a lot of noise out there from mostly from people saying, AI it's rubbish. It can't do this job. And, know, they don't say at the moment, they just say it can't do it. They sort of they leave the implication to you. If I want to learn to swim, I don't go to all the people who can't swim and say, how do you swim? to ⁓ to the people who can swim and say, how do you swim? Right. So the people that say, ⁓ can't do this job at the moment. ⁓ sort of irrelevant. ⁓ relevant people, the interesting people, the people you can learn from a people are saying, this is what I've done and it worked for me. That's not the same as it will work. identically for you, but there's something there if someone's found something that can work. And Martin and many others have got something that can work here. Where there's a meaningful step up compared ⁓ where we ⁓ not that long ago.
Anne Currie: Yeah, he said that himself, and he thinks have changed quite radically. the interesting thing here, and you did get mentioned in the couple of times actually in the Martin podcast, because around kind of stuff that he's been building to try out, ⁓ the of stuff that you used to be responsible for delivering ⁓ with gigantic teams of super highly ⁓ trained engineers like SIP stacks ⁓ How does that make you feel?
Jon Berger: It's brilliant, isn't it? I mean, just to be clear, I think when you say gigantic teams, mean in the modern word of gigantic meaning there was more than one person on the team. It wasn't just one person and some AI. I've always worked in what for most of the software engineering industry would be considered sort of smallish teams, sort of five to eight, which is a nice size for minimizing ⁓ pain in terms of team churn and in terms of communications overhead and just sort of like the coordination overhead of large teams. But that may well seem gigantic for someone like Martin, whose team is him and as many AIs as his credit card will allow. ⁓ I think it's fascinating and it's a major proof point because writing a SIP stack is non-trivial. It's not the hardest thing in the world, but it's more complex than probably most software, I would say. it does have performance requirements and it does have... But on the other hand, it's pretty testable. compared to some things, you know, and it is reasonably well specified, which makes it a good starting point.
Anne Currie: So I will bridge this thing. SIP stack, so SIP is a networking protocol. a lot of calls and ⁓ calls over IP, voice over IP runs on top of a protocol called SIP. that was kind of because you were ⁓ managing and leading teams of engineers that delivered networking software. a SIP stack was a kind of a fairly common piece of networking software to be developed at a period of time everyone was developing their SIP stack and that was a period of time in which we were quite active in, ⁓ well, very active in the tech industry and in the networking industry. Is that correct?
Jon Berger: Yeah, mean, SIP is a protocol used between entities that wants to communicate ⁓ to essentially negotiate how that communication is going to happen. So it's what we'd call the control plane. It's about how the call is going to work rather than the data plane, which is the thing that actually supports the call itself. Although Martin said he'd also done that as well. So there you go. Why not? Because you had a spare five minutes. What you didn't yet talk about with Martin and the obvious question is, so if one interpretation of what Martin said is software engineering is solved, great! ⁓ mean, there is something we should come back to there, which is that his new his new flavor of software engineering, actually. But software engineering is solved. Great. ⁓ Well, OK, a lot of people be going, well, wait a minute, but what about? And and there are a lot of what abouts, but to me, the big ones are sort of the maintenance and operability side. Like, OK, I as a one man band can create some software that works for me straight away. And it sort of basically works. And I am both the engineer and the customer. Maybe that's the future of all software. All software will have exactly one user, and the user is the person that says to their friendly AI, please write me some software that does whatever I need it to do. And there is no scalability and limited performance requirements for most software other than. the AI itself and whatever networking is required to allow you to talk to it, because all software is bespoke. That's one possible world we might end up in. ⁓ But I doubt it. You know, I can't really ever see my mom doing that. ⁓ So there will be still be a software industry in which some software is is sold to other people. Some level of service maybe is sold to other people. And therefore there is someone who's receiving that cash who's on the hook to ensure that that service works. And I think that's one of these get good at change areas, which is at some point in the next 10 years, almost all of that work maybe is done by an AI for you or a series of AI agents. That isn't true today. No one, think, is so convinced about the correctness of the AI generated software. that they've handed over the entire operations side to an AI to just go, yeah, please run my business for me. Let me know in a year's time when it's time to cash check. That's not yet. So there's a whole series of business processes and team interactions here that needs to evolve along with what the technology can do. Because they don't evolve along with what the technology can do, then anyone who is evolving alongside it is going to eat your lunch. They're going to go faster than you are. But if you if you evolve too quickly, then maybe there's a there's a challenge there. But that's where getting good at change is going to happen.
Anne Currie: ⁓ Yeah, because I mean, it seems so long ago, but it's like two years ago, everybody reckoned the new skill would prompt engineering. And now we know that the best way to produce a good prompt is just to ask to do a basic prompt into chatgpt or whatever to give you a good prompt for what you're attempting to do. And it's so much better and it'll give you back a really good prompt.
Jon Berger: Thank
Anne Currie: prompts engineering is dead. The AIs just much better at prompts engineering than we ⁓ are.
Jon Berger: Well, they're both better at it in that sense. You can ask it to write prompt, but also just the thinking models are better at just doing that as they go. can give them something simple and you can see them creating their own prompt as they're going.
Anne Currie: Yeah, it is quite astonishing. ⁓ Yeah, the time when which you write, you carefully craft your own prompt is very much at an end. For most of them, you should be asking that the chatbot will help you do that, we'll give you a first pass.
Jon Berger: Yeah. Carefully crafting your own prompt is still a good idea alongside AI as opposed to doing something on your own. It's just one of that. That is one of a whole class of things where you probably want to try using AI alongside to help you.
Anne Currie: ⁓ I one of the interesting things in Martin's, ⁓ when Martin was describing how he learned to ⁓ program differently in order to Claude and Codex, was when he said, you've got to define what good looks like upfront. The irony is that all my books and everything had been about how progress has happened through trial and error and iteration. And he said, well, the problem with that is that you can't do trial and error really, because it's just going so much faster than you. You need to say, well, I think I want something where good will look like this. Go for it. And then at the end, go, did I? Well, let's refine that and go for it again. Which could say a different form of trial and error, but it means you have to think in much bigger chunks.
Jon Berger: trying to narrow a different scale. Yeah, you've still got to have a sense of what you're trying to achieve, which as Martin said, was always good management anyway. It's like, ⁓ and build me something and then I'll tell you whether it was good or not. ⁓ not a good place to be. ⁓ it's still not a good place to be. And now, and now AI can call you out on that without anyone feeling bad.
Anne Currie: you You Yeah. Yeah. There's an excellent Mitchell and Webb look sketch. Not the one about are we the baddies, which obviously everybody's seen millions of times. There's an, would say there's an even better one about ⁓ a creative trying to get a good brief from an exec who basically cannot brief at all and is trying to basically constantly going, ⁓ time you produce something, it's like that, but not like that, obviously. ⁓ like that, but not like that. their It's a terrible way of getting stuff out of creative humans. And Martin's point is, it's a terrible way of getting stuff out of AIs as well. ⁓ It's your management skills that you need to be sharpening up
Jon Berger: Yeah, it is. But I wouldn't want people to think, well, you've got to go away and spec the whole thing in advance before you can achieve anything, because AI is really good at the spec inside as well. And and in the helping you to refine your thoughts. Now, even if it's just a rubber duck where you say you are, want something like this and it and just by hearing yourself say it, you go, ⁓ wait a minute, that wasn't very clear or.
Anne Currie: You
Jon Berger: actually, no, don't want that or I can add some more or whatever, but it's, it's, you can use AI to do a lot of the legwork of that spec. And in fact, it will result in much better quality output. If you say, well, this is the sort of thing I want. What, what, what, what questions do you have? Can you write yourself a spec and review it and ask me about any questions you have and And you you work with the the that first level agent on the spec before you then move on to the actually now go and build the thing.
Anne Currie: Yes, it's not the throw over the wall at the AI. It was, ⁓ I think, one the most interesting things ⁓ on Martin's when was describing his new SDLC, his new AI software development lifecycle was, ⁓ kept saying we the time when he meant me and the other, ⁓ the chat. Well, was a confused by that because I thought, who's the other people? no, it's Claudian Codex. ⁓ I thought it was quite interesting and it quite amused me that he said, ⁓ ⁓ one wants, everybody likes to work in a team. No one wants to get rid of all ⁓ humans in their team. But it was also at the same time, quite clear that he worked on his own, although he's a very sociable person, he's a lovely, lovely guy. ⁓ He worked on his own quite happily with the chatbots and he referred to them as weak. ⁓
Jon Berger: And that's.
Anne Currie: Yeah.
Jon Berger: Yeah, I mean, it's a very positive sign, isn't it? That he uses the word we. One of the indicators of ⁓ a ⁓ less than great, let's say, software engineering organization is when the different teams do just throw stuff over the wall to each other, when product management just throw something over the wall to the engineering side, or if you have a separate test team. things get thrown over or operations. It's like, and you want all those people to be using we building things together because they will be more successful. But yeah, that was interesting. mean, the other part of that was I think there's been there's been angst in the software industry for. almost as long as I can remember that people that went into the business to write code, that was that was how they saw themselves. They wanted to write code. They enjoyed the writing of the code. Often they didn't really enjoy a lot of the other bits. They didn't really like writing specs. They didn't really like testing it. You know, all of those all of those other things. ⁓ But they they saw themselves coders and and ⁓ is not the valuable bit. It's always been an interestingly difficult bit. And there's always been an awful lot more people in the world that couldn't do it than that could, which is why we've made money out of that in the past. But software engineering as as Martin now views it is, well, my goal is to create something that does something useful. I don't really care who writes the code or who writes the spec or who does the testing or in fact.
Anne Currie: you
Jon Berger: Actually, I sort of do care. I'd really like my AI to do it all for me because that's a million times faster or more. And, you know, I don't have to do all the bits that I don't want to do. And he views it as we, as you say, we created this software. He's not even seen the code. And he feels really happy about that. But that's a great thing. He feels happy about that. Isn't that brilliant?
Anne Currie: So it's an interesting one to say, because he pointed out very correctly that we don't see the assembly language, we used to in our earlier careers, we were very familiar with the assembly language. ⁓ I haven't looked at a piece of assembly language in some decades. ⁓
Jon Berger: No. And how many times did you ever look at a voltage on a chip? Never. Never. Never did that. So I never really knew what was going on in this thing that people told me it was a computer, but might just be magic.
Anne Currie: That's true. Yeah, but I never referred to me in the compiler as we.
Jon Berger: You're just not a team player and that's the problem.
Anne Currie: Maybe it's because I don't build in Rust and the Rust compiler is much more sophisticated. Maybe the Rust compiler would be my pal and I would think about what the Rust compiler was doing while I wasn't around in the room. But the C compiler was never, we were never on friendly terms, we were never on we terms.
Jon Berger: Yes.
Anne Currie: Now, but so we've done half an hour and that was really interesting. And we are going to ⁓ have more conversations, I think, about Martin, because I specifically want to I want to start diving in on some of the things that I thought he said that were very interesting about the kind of although this might happen with you or it might happen with with somebody else who's more expert than either of us two on on this. I was really interested in his comments on on testing and how it expanded. the realms of what was a plausible testing like ⁓ multivariate testing for the edge cases on conditionals if tests, It was interesting that he was saying, well, you just wouldn't do that except on air traffic control systems. But now you can. That's going to change everything. we might talk again a bit about that. But there's so much here to talk about. I'll see if I can get a couple of Rust experts in and a couple of testing experts in. And we can just untangle this a little bit more. But thank you very much for this chat. And we are. As I say, you're very usefully cached. locally, so we will definitely have more discussions on this podcast about what we've just heard and what we're learning. But thank you very much, Jon. And see you again. See everybody see all the listeners and watch us to this podcast again soon on asynchronous and unreliable podcast.
Jon Berger: Thank you. That was fun.
