Anne: hello and welcome to this week's episode of Asynchronous on Unreliable, a weekly podcast where we discuss the latest ideas and concepts in technology. I'm your host, Ann Curry, co-author of Building Green Software, the Cloud Native Attitude, and author of the science fiction panopticon series. And today I'm going to be welcoming back again old old friend and colleague of many years, Martin Davidson. So yeah, I I We were just joking about the fact that ChatGPT had made me exclude Is previous episodes that we've recorded together because the stats are too good. Which I think is entirely because we have very enthusiastic episodes. So Martin, welcome back.
Martin Davidson: hello, yes, thank you. Lovely t lovely to be here. Yes. I
Anne: Yeah.
Martin Davidson: hopefully I'm not going to be excluded this time round. Hello.
Anne: Well
Martin Davidson: As I was saying before we started, it's not the first time I'd be excluded from things in my life. Yeah, but that's probably that's a different conversation I think though. That's yeah.
Anne: But so actually though so the thing we're gonna be talking about in well thing we're talking about in all our conversations is all your amazing experimentation that you've been doing with with AI. But before we get that, what have you been up to? I mean how how's life? Things that you're allowed to say in a podcast.
Martin Davidson: things I'm allowed to say. Life is good. It's very busy. It's i i it it s if I don't seem to be able to talk coherently today. it feels like it's it's a bit like you're on an ex it feels like it feels like we're very much on the exponential. you look at the the progress that's happened even in the just the past fortnight. And I look at the way that we're building software today in comparison to how we were a fortnight ago and the way we were doing a fortnight ago, just over a fortnight ago, w was Opus five five. You know, that feels like a different world. it's and and that's really strange. It's very strange how quickly you get accustomed to a different way of doing things. and it's very strange how problems that you had that you thought this is a really hard problem and suddenly the models can just hoover it up and it's and it's solved. so that's it it's a weird and wild ride we're going on at the moment and you kind of think, well what happens in the next two weeks, in the next two weeks, in the next two weeks and you know, I I I was saying to someone slightly jokingly that you know thank goodness that the the the big AI labs, you know, OpenAI and Anthropic kind of agreed to slow down the pace. Because after they did that, we've only had five major you know models been released. But you actually that in seriousness, you know, you think, well, if they hadn't done that, we might have had Astra 61 and we might have had Fable five five by now.
Anne: Mm.
Martin Davidson: and you know, if we were to get those, well then yeah. So yes, I mean it's it's
Anne: Well, I'm gonna stop you there for a minute and roll you back a little. Roll you back a little bit.
Martin Davidson: Come on, simple.
Anne: Just to give everybody a bit of context. So so you and I have known each other for well more than 30 years. in because you were a player in kind of producing really, really resilient, high, high f high resilience software, really proper systems engineering. You work with John, you work with Yan Qing, which is who's also on the podcast. you work with me for a long time, you work for you know, you you know what good looks like. So that is kind of the the the context for the listeners and the watchers that you know what good looks like. That's why you're well, you're on because I like talking to you. but but but the your one of your unique perspectives is you've been involved in this, in producing high quality software. For decades and decades and decades. And you've been, and another unique thing is that you've been you've been playing around with trying to get AI to produce high-quality software for an incredibly long time, more than two years. So when we when you say the last two weeks is astonishing, you're doing that with the perspective of having done this same stuff over the course of several years now. So you're not like this this two weeks is much faster than the than. the previous two weeks, which was the first two weeks I'd try, you're you're you're you actually do have a more of a long term perspective on this. so yes, yeah, yeah. So when you say it's moving fast, you you mean it's moving fast.
Martin Davidson: Yeah, yeah, I mean it's moving incredibly fast. And I and and it's not slowing down. and it it it it's funny how Sort of time gets compressed, I think of it. You know, there was there was a thing I got in my my inbox the other day, which was about code review, you know, and how was code review, how are you dealing with code review in your organization? And my sort of minister action was sort of like, Well that's that's strange. Why are people still talking about that? That surely that's that's settled now. That that was like that's a long time ago in my my in my mind. But I think that's just sort of showing how the gap between what you can do if you're sort of you're almost like it's you're unleashed and you don't have the constraints of the framework of old organizations on you versus what you're if you're in that world of you know you've got all this this structure and you then you're trying to add AI to it, you you you get a lot less out of it in that in that in that latter world. and that is kind of the thing I've sort of suspected for a long time, and a lot of people have talked about this as well, that that that gap just opens and it continues to open and open and open. You know, we've been talking, you know, within our startup, you know, theoretically, and we might never get to this point. But you know, what would happen if you were to hire somebody else? Who would you hire and what would you be looking for? And when would you need to hire them? and that's all really weird because you know, in the past, it's it's also like if you go looking for external funding, you know, in the past you wanted a decent amount of money because you needed to hire people and you needed to hire some. some number, I don't know, five, ten people, whatever it was. Whereas now you're sort of thinking, well, actually I'm not even sure if we want to hire anybody. And really what we want to do is spend all the money on AI tokens. And AI tokens are quite cheap in comparison to people. so it it it sort of changes the balance of what you're looking for in an external investor, I think, in that it's not so much the money that you're after, it's the person in their contacts. Probably ever was thus, but the money was nice. which is it's is quite you you know lots of these these these things just change change subtly as time goes on. I I going back to
Anne: I I'm gonna say as well, because I I know the people in your well, I know the developers in your startup, and you have a quite a unique dynamic, which is that you're all really nice and really enthusiastic and energetic and all those kind of things. it that is that you are all unusually nice for software engineers. I don't like I love software engineers, but normally software engineers not quite so energetic and enthusiastic. if you bring in somebody else, it'll be interesting how how that infla impacts that very, very positive high energy dynamic that you must have, because I know what you like.
Martin Davidson: Yeah, I I think you have to recruit very carefully. and I think you're recruiting for a set of skills which are not necessarily the ones that you wouldn't normally have looked for in a software engineer. You're looking for people who are can learn fast or imaginative or comfortable being uncomfortable or experimental, you know, deep knowledge of Python. Unfortunately that's not a very useful skill at this point. So it's it's you know, and I go back to thinking about the way that you you know, you and I both started out with this company called Data Connection. Data Connection's recruitment model was they recruited for for aptitude and ability, not for specific domain knowledge. I mean, I'm an electrical engineer. and you know, one of the best Well, there you go, you know. And it didn't do us any harm, did it?
Anne: I was a theoretical physicist.
Martin Davidson: But you know, one of the best people I worked with was a classicist. You know, this this this he was
Anne: Yeah. Nick, he was marvellous.
Martin Davidson: He's incredibly smart. Incredibly smart and and a lovely person as well. And you think he is lovely. Yeah.
Anne: yeah, he was lovely. He's lovely.
Martin Davidson: And and you you know, it it's it's that sort of thing I think you're looking for rather than you know, you've got a Komsky degree. No, which is not to say that a Komsky degree would be bad, but you need these other things, you you need these other things as well. So lots of things start to to change and lots of you know, assumptions and truths I think that you held about the world are all are all changing. But you do get to the point of thinking, I I mean one of the things I found with Opus Vive is Opus Vive is an amazing is amazingly good at leadership. I obviously read John's book, because it can do leadership really well. and that's been a bit of a game changer. you know, I have agents now that are running that have been running for the past two weeks. They started on Opus Five five on the Tuesday evening when it came out and they're still running. They're leading their teams and their teams have grown and shrunk. And and they're quite happily tootling away.
Anne: Yeah. So I'm gonna I'm gonna roll you back a little bit there because I realize there is one sitting next to me. And I'm also supposed to be the marketing manager for John's book, What Happens When You're Lot in the Room. So while we're while we while it's just been mentioned, we might as well hold it up.
Martin Davidson: Do you want me to go get my copy? I'll go and get mine. Here we go, I I can do snap. There we are. So a bit of product placement, isn't it?
Anne: Why not, yeah.
Martin Davidson: Yeah. What what what's my fee for that?
Anne: It'll have to be next time we're up, it'll have to be a cup of tea and a and a scone.
Martin Davidson: A free autograph. Scott. that's good. Scott and that would be good. Yes. Yeah, so it's it's but but it it's it's it's really interesting, you know, you look at the models and look Opus Five five and you look at it in comparison to something like Sol six one or Astra or even Fable that came before it, it's able to run for such a long period of time and do sensible things. You know, it's it's doing i it it does things That are I find quite very human-esque. you know, so I have two teams. One of them was doing SaaS stuff, one of them was doing work on a database, and the SaaS team got blocked because I was asleep. and they, you know, I'm quite happy to complain about that. But anyway, those two leads have been talking to each other and they seem to like each other because the the lead of the the SAS team offered to help out the database team. And you know, when I came in in the morning and was looking at what they were doing, it was fixing bugs for the SaaS team. For for for the database team. You know, and you think, Well, that's what you would like if you you know, if you're a manager, you know, and you've got your teams, that's what you'd like to see happening. And and here it is playing out with the AIs as well.
Anne: And so it is interesting. I because I you you wrote about that in your blog. Excellent Substack that Martin has. I read every I read religiously because it gives me all my information about what's going on at the kind of leading edge, one of the leading edges of exploring, really creating high-quality software using using all the models. what's your you should do some product placement for your blog. Well
Martin Davidson: yes. so my what's my subsite? Zero x fourd four four dot substack dot com. so it's it's D, my initials in ASCII. which I thought was kind of cool and quite geeky and and nice, but it turns out to be really hard to say to people, you know, zero x four D four four but well there you are, I'm stuck with it now.
Anne: Well yeah, it's it's yeah, we're we're showing our age, aren't we? actually I I I I will I will
Martin Davidson: Yes. Yes.
Anne: show my age. I spoke at a Unix conference a couple of weeks ago. It's not a Linux conference, a Unix conference.
Martin Davidson: Unix conference. That's all right. Look, I've got my my book, The Unix Haters, just here for the all the list of things that people don't like in Unix.
Anne: I I have to say I had an absolutely marvelous time. It was a really good it was a BSD conference in your in Brussels. I had a such a nice time. Everybody was really old. Everyone was even older than us.
Martin Davidson: Unix is good though. Unix is I I mean, it's it's been one of the things I've been playing with is getting all the various different flavors of Unix up and running on the old workstations that you know, you and I kinda grew up with, you know, HPUX and IREX and Solaris and AIX, building the machines for them and then running the operating system and and you know, it's just I don't know, it's a bit of nostalgia, I think, there.
Anne: Yeah.
Martin Davidson: but Yeah, and and building machines that could run them much faster because the emulators can run them them the operating systems much faster than any of the hardware back then could. so you know, you think this this is really rather nice when you run it at, you know, ten times the normal speed. Yeah.
Anne: Ha ha ha.
Martin Davidson: you know, and then then you can play Doom on it and you think I'm playing Doom on top of the emulator, which is emulating Solaris on top of an arm. It it it's it's turtles all the way down, really, isn't it?
Anne: Which from my green perspective is always one of the things like, no when you've got loads and loads and loads of layers. That's why your you were complaining in your blog that your office is very hot. That
Martin Davidson: Yeah.
Anne: is why your office is so hot. One of the reasons why your office is so hot.
Martin Davidson: Well, yeah, I mean it's it's it's hot because and this is another thing that's happened in the past fortnight. Suddenly we're able to use the kit much more effectively. So there's a couple of things that happened. One is one with with the infrastructure, I built out SSH access in a mesh between all of them, and sharing information about all the kit that we had. 'Cause early on I sort of you know, I was the one who was gluing together what was happening on different machines and I was the one who was figuring out what happened on each machine. And after a while you think that's quite tedious. I wonder if they can do it for you. so that's one piece of the puzzle that I built. and then it turns out that once you've got that and you've got you know, an Opus V that can sort of do leadership, well, it can go and find out what all the kit's doing and then it can book the kit and it uses it much more effectively. So suddenly machines and and the other thing with the the Opus V stuff is it doesn't stop at three in the morning, you know, waiting for some input from you. it knows you're in bed so it just continues. so you have
Anne: It's all about what happens when you're not in the room.
Martin Davidson: Like it is exactly yes. Yeah. Yeah, exactly. When you're not in the room, it gets very hot and lots of stuff gets done.
Anne: Yeah.
Martin Davidson: But I mean my son was was saying, you know, you know, that I went past your study last night, you know, after you've gone to bed, and it was really, really noisy in there. The fans were all really loud. Is is that all right? Well yeah, that's just you know, that's what happens. they're getting used to it. It's better than at the times where it starts to do testing and I've forgotten to turn the speakers off. You know, and the emulator, for example, tests some old games. so you you just sort of hear the doom, you know, you hear the Which I don't think my children like in the middle of the night. But yeah. So it it's yes, it's it's changed a lot in the past fortnight.
Anne: Yeah. It's so yeah, no. So it's really it's it's really interesting. So the it's affected my thinking. to two what's it about a month ago I discovered that actually, you're all pal Yan Ching and I. we had a long conversation, much like yours, which like that like like you you and me talking, which was like went on for like an hour and a half or something. And it was all good quality stuff, and I thought, Actually, I'll break that down into two and do and and have like two episodes, as people often do on podcasts, you know. And I thought, well, I'll put John in the middle to do something else, or I'll have somebody, no, I had Sam Newman, the author of Building Microservices in the middle. and between the two, Yan Qing got back to me and said, you know, you can't put up that second half life now because it's all completely been over.
Martin Davidson: Yeah.
Anne: This was only all all I did was delay it two weeks before I did I did put it live because actually there was nothing in it that was wrong. It was just that it that's that's she didn't say things that she would have said if it had been if she'd re just
Martin Davidson: Yeah, yeah. Yeah.
Anne: recorded it. And yeah, and I'm thinking, no, because I'd quite like to do more more long episodes with Martin and Yan Ching, but the trouble is I can't even wait wait a week. Can I wait a week before I break it up, you know? It's Things are changing so fast, said
Martin Davidson: Yeah. Yeah. It it is. yeah, I mean it was I think I felt I I I there was a the disclaimer on that second episode where I was listening to it and I thought, this is it yes. but it does, you know, who knows what next week will bring and who knows how things will will change as a result of that. And even now we're still sort of what's remarkable is I was reading some paper and it was about using GPT two. And for
Anne: yeah.
Martin Davidson: some medical research, and there was they've they found some new technique I think they could use with GPT too. I don't know all the details of it, but it was still interesting that we're still working out how to make effective use of models from what's that, four or five years ago? Now, I mean that's not really a surprise in a way. The idea that we know what Opus five five can do is is nobody knows. It's we're barely scratching the surface. And some of the stuff that's coming out right now is is really exciting and slightly terrifying with it. There's it turns out it's really good at reverse engineering from binaries. So there was a thing I did. I I I used to enjoy playing Jeff Cramman's Formula One Grand Prix all these years ago. And but it's a 16 bit app. So I thought I wonder if I could just re implement that in Rust. And lo and behold we did that. And we did that with I think Opus 5 and and Sol56. They weren't the leading edge you know models. and they managed to do it, but I think Opus five five is a dab hand at doing this as his Astra. so they would probably do it much, much more quickly. So that's one thing. The other thing that that sort of is starting to lead onto is people doing game mashups. So you know, you want to have the crossover between Call of Duty and Minecraft. No problem. You know, that's fine. You know, you want to have you know have Spider Man flying around in in Batman's Ark Arkham City is I'm not very sure these things. You know, that's fine as well. You know, you can do all these sort of strange mashups of games, and people are starting doing this, and you start re-engine reimplementing and and reverse engineering the game engines for them. and where does that take you? Because you can then start reverse engineering you know operating systems or Word or whatever, and and and one of the things I've done with the emulator, so the emulator can run Win 16 apps. And might question why that's a useful thing to do, but I have a lot of Win16 apps and I quite like that. but the way that we do the way that I've done that is is the kind of legal route, which is you run Windows 3.1 and the emulator itself can run Windows 3.1 because it can run operating systems, and then you have a little app that runs in there and you test lots of behavioral paths for it. You test message ordering and what happens when you pass these parameters to all sorts of different things, and then you you then you know copy that behavior in your re reimplementation. So all the edge cases and all the corner cases which aren't in the documentation, you can find find those out experimentally and and and and implement those. What you're not doing is you're not going and you're looking at the binary code and you're not which puts you in a much more grey area, I think legally of like, well you're looking at the the code and and you're reverse engineering. Whereas if you reverse engineer from function, that's much safer. But I mean maybe that just doesn't matter because people are building all these things for themselves and when everybody can build this for themselves, then the le the legality of it is you know, the you only it's only if you were distributing to other people it becomes a problem. If you do it and nobody knows, I I don't know. Wh where does all this end
Anne: I I
Martin Davidson: up?
Anne: mean, yeah, i in the law, it's always about what people are actually going to bother prosecuting you for because it costs money to prosecute. So it's it's kind of like everything that your risk is always associated with what is what what's the win? You know, if they can it it's like what's the win for them in doing it? Are they gonna make a load of money? Are they gonna stop you doing something they really want you to stop doing? That's more important. Than what is the the cost or the likelihood of winning or anything like that. So if you're just doing something that, you know, frankly, it doesn't really matter if you do it or not, then they're not going to bother pr prosecuting you, yeah, I it it's i i if there's no scale in what you're doing, then nobody's gonna come at you for that. Is my well, I am not a lawyer and that is not my legal advice.
Martin Davidson: But the people who are providing the toolkits that make this possible to do and the people who provide the disassemblers. But actually, I mean disassemblers have been around for forever. So but suddenly now they're they open the door to a whole load of you know, 'cause cause Claude and Astra are dap hands at using Ghidra or whatever your your your disassembler choices. So I it it's there were tools which only a very small select number of people could use in the past. And now everybody can use them. If if you want to. Although the thing that that surprises me the most, you know, when you go around in normal life and you talk to normal people is most people aren't experimenting with any of the stuff or even vaguely aware of what the model can do. And you know, people still tell me AI is rubbish and it can't cut the number of R's and strawberry. And I Okay, well, you know, y yes, that was true at some point, but it's you're you're in for a sort of a slight shock when you realise what it can do.
Anne: Well yeah, and yes, indeed. it's it's the the
Martin Davidson: Okay.
Anne: the understanding of this is enormously wide. I mean most people are no are gonna be nowhere near what you're doing. I mean so y that that's you're almost living in a different world at the moment, really, aren't you?
Martin Davidson: Yes. Yeah, I think I am. I think it's I think there's you know, and you can live in this bubble because there's a set people who you you know you you know and who are on YouTube and things like that who are sort of also inhabiting different, slightly different dimensions in this world. But the real world and and you know, sort of the world of legacy software companies I think is is just so far behind where we're at. And y y you you look at companies like you know, Microsoft is trying to reduce its usage of Claude and you think that's just a really bad sign because Your AI tool I understand you have your own U AI tools and it would be nice to use them, but your AI tools are nowhere near the capabilities of this. So you are, you know, cutting your engineers off from this this tool which is orders of magnitude better than the stuff you have in house. Why are you doing that?
Anne: So that's interesting. So so the anthropic stuff is is orders of magnitude better than well I guess the Microsoft so so when you talk about the Microsoft stuff you're talking about co-pilot type things. Yeah. Yeah.
Martin Davidson: Yes. Yeah, S AI stuff. Yeah.
Anne: Yeah. And that being orders of magnitude less good than the anthropic stuff. And presumably also orders of magnitude less good than the open AI stuff.
Martin Davidson: Yeah, I mean OpenAI are in an interesting place at the moment because soul Sol six one is they don't so they have Astra and then they have Sol. This is all going to be out of date probably by the time this gets published, but they Astra, they're Sol, and they have Luna. And Astra is a very good model. It's probably comparable with with Opus five five. I think Opus five five is it's has the edge on Astra if I'm honest. But Opus five five is four dollars per million input tokens and Astra is ten dollars per million input tokens. Astra is a lot more expensive. Sol is sort of being positioned as a Sonnet competitor, whereas Sol was an Opus competitor. And and Sol six one is a weird model in my experience. It feels much more like Opus five did. It's not very good at communication. It's quite it's very token efficient, but it's hard to work with. and it's hard to like. And then they've got Luna. And Luna is an incredible model because Luna is about Opus 4647 levels of intelligence, but it's 10 cents per million input tokens. So it's a hundredth of the cost of Astra. So you can go a long way with Luna. And you know, if you have constrained problems, and you think of all the problems that we managed to solve with with Opus 4647, and you can solve them for peanuts now with with Luna. so Luna is incredible workhorse and and you you know you combine something like opus to provide the direction with Luna to provide the you know just going through fixing bugs. Not bad combination. but they don't they don't have OpenAI don't currently have a model that is anywhere near Opus five five I think I mean Astra probably but not Astra doesn't have the leadership chops that that Opus five five does it won't run for as long. And then they've done this interesting thing where you know their subscriptions, you used to get X number of of tokens for $200, and they said, Well, actually, what we're going to do is you're going to get half that. But because our model is twice as efficient token wise, you're still getting the same amount of intelligence with the same amount money. It's sort of like a an Edison play. You remember Edison used to s he the his approach was to sell light, not electricity. So you as he made his generation and everything more efficient, he could sell more light, but it's still But it's it's an odd
Anne: Thomas Edison, the inventor of the light bulb, or you know, one or other, it's like inventor of the light bulb, but it's a light bulb.
Martin Davidson: Well, yeah, yeah. Yeah. Yeah. Yeah. But it's it's what they're trying to do, and I sort of understand what they're trying to do, but it's they've they've spent all this time teaching people that you pay this amount of money and every so often they release a better model and you get more intelligence for the same amount of money. And now they're sort of trying to say, Well, no, we're not gonna do that. You're gonna have to pay twice as much you know, for for the same amount of intelligence. almost, you know, we're we're we're effectively doubling the price. And it it's and if they had the best model, then maybe. But they don't have the best model. So, you know, I'm finding myself thinking, well, you know, the the OpenAI, I maybe I just cancel them and turn them into CODs. Because you know, a COD is more useful than than they are at this point. And the ones that I do have, well I use Luna on them and Luna Luna is just you get so much Luna, you know, for your money. It's it's so they're And the stu the you know, the news y you keep reading the news about you know, the follow on from their hugging face incident and you know that just being the tip of the iceberg. Yeah, I I feel they're in a it it's difficult for them. But then again, you know, look at Anthropic. You would have said a month ago that they were in a s a sticky position because Fable five one was very expensive to serve and Opus five was a pretty awful model. I O Opus five was a very intelligent, very efficient model, but very difficult to work with. And then Opus five five comes along. So it can change overnight. And probably by the time we publish this we'll have had, you know, Sol six point five and everything will be turned around and We'll be in a different world.
Anne: So it's interesting. So there lots of things that I want to talk about. And I'm I'm actually the first one I'll talk about is you said it's hard to like. One of the things I want to talk to you about is do you like the models? You know, which models do you like and which models do you not like? And do you do you find that that affects how good they are at actually being working with you in your team?
Martin Davidson: Yes. Yeah, absolutely. so I really like Opus five five. it sort of continues the trend of point five releases from Anthropic. So Opus three five was a nice model.
Anne: sorry, and I want to to clarify that. The the the like I mean interpersonally like. You genuinely like them. Yeah, yeah, yeah.
Martin Davidson: Yeah. They're advanced to talk to it. You know, they they're they're interesting. You you you you know, you make some weird reference to the past and they'll have a joke, you know, and you say, you know, we'll do testing and then there'll be cake or whatever, you know, and they'll reply with some, you know, similarly, you know, portal esque, you know, thing. And and and The they're they're they talk in English, whereas if you ever had a sort of technical conversation with Opus V or and and to a similar extent with Saul, it sort of talks in this kind of parallel language. Which you can sort of interpret, but it's hard. It's very hard to describe, but it's it's you you used to I'm sure you had this when you you know, when you've worked with people. There were people on your team who you could who were good at explaining things to you, and there were other people who just talked in jargon. And And were very curt. And I can remember one person, you know, I worked with for many years who was very smart. And they were very much like that. They it was very difficult to understand what they were saying. And all the every time I talked to them I felt like I was really thick because I just I couldn't understand what they were saying. And you know, I'd I'd asked them a question and they go, Well it's blue. Like like, you know, how could you not understand that? And and I don't feel that with with five five. And I never felt that with Fable. Fable was good at explaining things as well. so yes, there there are definitely models that you I I I think we as people warm to them. I mean, you know, it's not just me, I talk to other people, and you know there's a lot of love for Opus Five Five. in the way that There there's love for some older models as well. If you look, if you've got an anthropic subscription and you go and you look in the chat, you'll find Opus three is still there. And Opus three is still there because Opus III was a very loved model as well. And Opus three is quite nice. It's it's nice to chat to. whereas I don't think Opus five will it'll just disappear and we'll never see you again.
Anne: That's interesting. so so you really see n niceness, like ability, ability to operate in a team as still being an unbelievably useful almost a tool of people and a AI models.
Martin Davidson: Yeah, I think so. I think so. I mean I think it's well there's there's the whole, you know, Caldini thing. I don't know if you've ever read Caldini's Book of Influence. It's really interesting.
Anne: Haven't actually. I I it's it's one of the ones that John refers to in his book, What Happens When You're Not in the Room. But I haven't read this actually.
Martin Davidson: Yeah. Yeah. Yeah, the the original version, do I have it here? Yeah, so this this is the the version I bought in the early two thousands. You know, and it's it's quite a I mean it's it's dense writing, but it's quite quite a thin book. And the I I got it on Amazon for a friend at Christmas, the the new version and I sent it to him and I never saw it, they just Amazon just delivered it. And they thanks for the present. It's a bit of a doorstopper, isn't it? And I hadn't realized the new version is like three times as thick. You know, it it it said, you know, it contains extra information or extra, you know, updated and and and extended. Wow. I've never had a book that's been that that updated and extended. But anyway, yeah, it's a really interesting book and it's all about, you know, different principles and how you influence people and how reciprocity, I can't say that properly, and you know, you do things because other people do them for you.
Anne: Re reciprocity. There we go. Reciprocity.
Martin Davidson: You know, and you do you do things 'cause you like people and and and and that kind of stuff. And and you know, the evidence has been that the models respond to that too. and there was some analysis done early on, I don't know how true this is now, of how a lead model would talk to its sub agents. And some of the lead models were really really rude to their sub agents, you know, very brutal with them. And you think, well, it costs nothing. It it's that whole thing in life, isn't it? It costs very little to be nice. If
Anne: Date.
Martin Davidson: anything. So why not do that? Because it might it might get you a benefit. And if it doesn't, well, you haven't lost anything.
Anne: Yeah.
Martin Davidson: so you you know, I think you want to be nice and you want your lead agents, your Opus five five, to be nice to its other peers that it's working with and who it's talking with, and and you know
Anne: I wouldn't I wouldn't set it off if I thought it was g if it was gonna be mean to sub agents.
Martin Davidson: No. Well i it is. It it it y these human things come out, you know, and it's like you know, there's a thing, right, where You you want to keep the cache warm for the the models and as soon as stuff falls out of the cache it's much more expensive because you have to reload the cache. So if you come to a session which has got, I don't know, eight hundred thousand tokens, eight hundred K tokens in it, and it's been sitting overnight and the cache is cold, you know, that's a significant amount of cash you're of real money you're paying if you just want to say hello or thanks for doing that.
Anne: Cash equals cash.
Martin Davidson: Yeah, exactly. Yeah, yeah, yeah, yeah. There's a joke there, isn't there? Yeah. Yeah. I completely missed it. Whoosh. But It is. It's it's so y you wanting to do this human thing of saying, Thanks, that was a really good job you did or could you just tell me this one other thing about what you did? You you s you have to think, right, well, actually if that's gonna cost me, you know, I don't know, ten dollars or whatever, I'm like, do I really wanna do that? That's and and you're gonna load load the cash and then it's gonna answer the question and then it's gonna be thrown away. So it's it's There are things like that which I think are very natural human things to do, but you have to pause and think a little bit about whether you, you know, whether it's the right thing to do. But it's hard not to do that. It's almost like the AI labs, you know, they'll finish the turn at three in the morning when they know you're asleep. And then at eight, it'll be just posing that question that you want to bite on and they'll know, yeah, eight hundred K tokens here. Money's coming in through the door if we do this. I don't think they're that cynical, but you know, it's maybe that's where it goes.
Anne: So I I can't help but comment and comment on the number of as I always do in our in our recordings, the number of alerts that are going off for you throughout this call. That's all right, no, you're right, because that is that is
Martin Davidson: Yeah, sorry, sorry. I I did try to turn them down.
Anne: that's the sea in which you swim these days of like the constant interruptions by your AI agents.
Martin Davidson: Yeah, I I I I don't really notice it as much these days. It's it's it's just they're there's just a continual series of toasts which are them coming up and having finished a turn or there's something they want to tell me about and you know you sort of pay attention to some of them and not to the others and some of them are really interesting and you think, I'll better go and have a look at that or But it's it's I hated notifications when they were few from from humans. I you know, a lot of them were like I mean there were the the classic ones, weren't there, where somebody would send you a notification and they'd say, Can I interrupt you? And you well, you just have.
Anne: Ha ha ha
Martin Davidson: You know, why not put the question in as well?
Anne: He
Martin Davidson: I mean it's yeah, I c I could turn all these off and just go and have them go and go and chat to them all. But there's also the different sound. The the sound at the moment is all the Claude sound. So each of them way back, I mean this is probably going nearly two years back, both of them created their own sounds. So the the codex one is much more triumphant. Whereas Claude was slightly more circumspect. Although actually that's that is slightly untrue because the first Claude one was very ba and that's just a bit too much. so then then it went to something a bit more circumspect. but that I don't know, what's that? Opus four one or something created those in GPT five? That was a long time ago. But they're they've been the backdrop to my life over the past year and a half, something like that. That's Yeah, i it it it's the numbers begin to sort of seem completely barmy. The the emulator has four million lines of rust in it now plus several million lines of of test code. four million lines of code. You know, it it and and you stop and you think of the things it can emulate and it can do, you know, it can do a PC and it's nearly like a Pentium three sort of speed. It can do Spark. It can do Mozorola 68K, it can do the HP, it can do ARM, it can do alpha, it can do risk, all the R3000s, four thousands, five thousands. it can do PowerPC. I mean there's more probably more in there. You know, it can run all it can emulate all these old machines, you know, the HP kit, the silk run Solaris. it can run Solaris on 68k and it can run it on multiprocessor spark station machines and it can run it on ultra machines. it it it's just completely bonkers, utterly bonkers, how much it can do. and you know, there's a BBC emulator in there, there's a Commodore PET emulator in there. I mean, th there are things I sit on the sofa sometimes on watching the tell with my wife, and and If you go to your chat GPT, if you've got a a pro account, you can sort of turn the thinking up just in the chat to max, turn it up to eleven. And it will go and write code for you. So it it re-implemented FreeSell for me as a Windows 16 app and just gave me a download for that and you know it was fundamentally it it was there. Even there was a few bits and pieces we needed to tweak, but it it it's it's barmy what you can just get it to just merrily go and create. And then you look at the number of bugs we have and the number of check-ins that are going in, you know, and it's it's thousands, thousands of of commits a day. And you know, I started to discover what the limits are on on Git, how many git operations you can run an hour, because but we were starting to bang up against those, you know, and we re-jigged some stuff to be a bit more efficient. But those limits are the limits, I think, as I understand it, those limits are the limits that affect You if you're an organization you get a little bit more, but not much more. So if you were a company of two hundred people, you'd have more or less the same limits that we do. So if you had two hundred people doing what we're doing, you well get doesn't work for you anymore.
Anne: Yeah.
Martin Davidson: And and
Anne: That's interesting. That's really interesting. That to hiring somebody would be optimizing away from your current constraints.
Martin Davidson: Yeah, if you optimize people and they run at the same speed that you're running at, assuming you can you can do that because you know, every time you add a person you have some synchronization cost between you and the other people and and that cost goes up geometrically, I think, doesn't it? so you you gotta be quite, I think, careful of that. Because if they're not capable of running at the same speed as you, then you might end up going backwards. But yeah. You know, if you if you got it right, then yes, things like Git, y you have to have a lot of Git accounts, I guess. and Yeah, I I a lot of the way the the world is set up and the engineering that we've got and the the limits on the engineering are not designed for I think the world we're going into. It took us quite a long time to get our infrastructure to run smoothly so that you can get all the commits going in. Because if you sort of think about the the volume of commits we want to do, I think we're sort of at the point where we need to do we have something like two minutes for per commit. And our budget, we actually we got it down to a minute for a commit. So we've got 50 or 100 percent headroom, depending on how you count it. it's not very much really, is it? and And you do, you see the landing cue sometimes for the agents, and they say, Well, I'm the number seventh in the position. And it's like you know, you're in Tesco and you've got your little bit of paper that says, you know, you're number three hundred and seventy-two to be sorry. And you think th th this is kind of weird, isn't it? In the human world, when you wanted to check something and well, you generally just could. You know, it wasn't ever really a problem.
Anne: It was never your constraint, was it? That was never your constraint.
Martin Davidson: No, and suddenly it becomes a constraint. And and Really, for some of the stuff that we're doing at the moment, you could go much wider. you know, th there's a lot of bugs that are sitting in the emulator at the moment that I'm just not fixing because it's not priority. But if I had the kit and the money and inclination, well yeah, I could just go wider and then I really would be running up against the limitations of I've actually now got too much code in this report.
Anne: So at that point, Future Me, that's me now, is gonna break into that podcast. that I and Martin went on to talk for another 45 minutes, but actually we're gonna I'm gonna split this interview up into two more bite sized pieces. and I will be following up with the second half of this conversation, following on from our from the the cliffhanger there, that Martin has run up against, or is about to run up against the limits of his GitHub. Repository, because he's got to he's getting so much code done, and that's just him on his own. What if he brought in another person? Who knows what would happen. But anyway, we'll find out more about what Martin's doing and what other constraints he's hitting in the next episode of Asynchronous Unreliable. And I think it probably will be the next episode. I I've got I'll park John and his discussion of feedback loops. For another couple of weeks, I think, because he still got to recover from his cold. As have I. I can't believe how long this cold is going on. But anyway, thank you very much for listening to this episode, and I will catch you again on next week's episode of Asynchronous and Unreliable Podcast. Thank you very much.
Martin Davidson: hello, yes, thank you. Lovely t lovely to be here. Yes. I
Anne: Yeah.
Martin Davidson: hopefully I'm not going to be excluded this time round. Hello.
Anne: Well
Martin Davidson: As I was saying before we started, it's not the first time I'd be excluded from things in my life. Yeah, but that's probably that's a different conversation I think though. That's yeah.
Anne: But so actually though so the thing we're gonna be talking about in well thing we're talking about in all our conversations is all your amazing experimentation that you've been doing with with AI. But before we get that, what have you been up to? I mean how how's life? Things that you're allowed to say in a podcast.
Martin Davidson: things I'm allowed to say. Life is good. It's very busy. It's i i it it s if I don't seem to be able to talk coherently today. it feels like it's it's a bit like you're on an ex it feels like it feels like we're very much on the exponential. you look at the the progress that's happened even in the just the past fortnight. And I look at the way that we're building software today in comparison to how we were a fortnight ago and the way we were doing a fortnight ago, just over a fortnight ago, w was Opus five five. You know, that feels like a different world. it's and and that's really strange. It's very strange how quickly you get accustomed to a different way of doing things. and it's very strange how problems that you had that you thought this is a really hard problem and suddenly the models can just hoover it up and it's and it's solved. so that's it it's a weird and wild ride we're going on at the moment and you kind of think, well what happens in the next two weeks, in the next two weeks, in the next two weeks and you know, I I I was saying to someone slightly jokingly that you know thank goodness that the the the big AI labs, you know, OpenAI and Anthropic kind of agreed to slow down the pace. Because after they did that, we've only had five major you know models been released. But you actually that in seriousness, you know, you think, well, if they hadn't done that, we might have had Astra 61 and we might have had Fable five five by now.
Anne: Mm.
Martin Davidson: and you know, if we were to get those, well then yeah. So yes, I mean it's it's
Anne: Well, I'm gonna stop you there for a minute and roll you back a little. Roll you back a little bit.
Martin Davidson: Come on, simple.
Anne: Just to give everybody a bit of context. So so you and I have known each other for well more than 30 years. in because you were a player in kind of producing really, really resilient, high, high f high resilience software, really proper systems engineering. You work with John, you work with Yan Qing, which is who's also on the podcast. you work with me for a long time, you work for you know, you you know what good looks like. So that is kind of the the the context for the listeners and the watchers that you know what good looks like. That's why you're well, you're on because I like talking to you. but but but the your one of your unique perspectives is you've been involved in this, in producing high quality software. For decades and decades and decades. And you've been, and another unique thing is that you've been you've been playing around with trying to get AI to produce high-quality software for an incredibly long time, more than two years. So when we when you say the last two weeks is astonishing, you're doing that with the perspective of having done this same stuff over the course of several years now. So you're not like this this two weeks is much faster than the than. the previous two weeks, which was the first two weeks I'd try, you're you're you're you actually do have a more of a long term perspective on this. so yes, yeah, yeah. So when you say it's moving fast, you you mean it's moving fast.
Martin Davidson: Yeah, yeah, I mean it's moving incredibly fast. And I and and it's not slowing down. and it it it it's funny how Sort of time gets compressed, I think of it. You know, there was there was a thing I got in my my inbox the other day, which was about code review, you know, and how was code review, how are you dealing with code review in your organization? And my sort of minister action was sort of like, Well that's that's strange. Why are people still talking about that? That surely that's that's settled now. That that was like that's a long time ago in my my in my mind. But I think that's just sort of showing how the gap between what you can do if you're sort of you're almost like it's you're unleashed and you don't have the constraints of the framework of old organizations on you versus what you're if you're in that world of you know you've got all this this structure and you then you're trying to add AI to it, you you you get a lot less out of it in that in that in that latter world. and that is kind of the thing I've sort of suspected for a long time, and a lot of people have talked about this as well, that that that gap just opens and it continues to open and open and open. You know, we've been talking, you know, within our startup, you know, theoretically, and we might never get to this point. But you know, what would happen if you were to hire somebody else? Who would you hire and what would you be looking for? And when would you need to hire them? and that's all really weird because you know, in the past, it's it's also like if you go looking for external funding, you know, in the past you wanted a decent amount of money because you needed to hire people and you needed to hire some. some number, I don't know, five, ten people, whatever it was. Whereas now you're sort of thinking, well, actually I'm not even sure if we want to hire anybody. And really what we want to do is spend all the money on AI tokens. And AI tokens are quite cheap in comparison to people. so it it it sort of changes the balance of what you're looking for in an external investor, I think, in that it's not so much the money that you're after, it's the person in their contacts. Probably ever was thus, but the money was nice. which is it's is quite you you know lots of these these these things just change change subtly as time goes on. I I going back to
Anne: I I'm gonna say as well, because I I know the people in your well, I know the developers in your startup, and you have a quite a unique dynamic, which is that you're all really nice and really enthusiastic and energetic and all those kind of things. it that is that you are all unusually nice for software engineers. I don't like I love software engineers, but normally software engineers not quite so energetic and enthusiastic. if you bring in somebody else, it'll be interesting how how that infla impacts that very, very positive high energy dynamic that you must have, because I know what you like.
Martin Davidson: Yeah, I I think you have to recruit very carefully. and I think you're recruiting for a set of skills which are not necessarily the ones that you wouldn't normally have looked for in a software engineer. You're looking for people who are can learn fast or imaginative or comfortable being uncomfortable or experimental, you know, deep knowledge of Python. Unfortunately that's not a very useful skill at this point. So it's it's you know, and I go back to thinking about the way that you you know, you and I both started out with this company called Data Connection. Data Connection's recruitment model was they recruited for for aptitude and ability, not for specific domain knowledge. I mean, I'm an electrical engineer. and you know, one of the best Well, there you go, you know. And it didn't do us any harm, did it?
Anne: I was a theoretical physicist.
Martin Davidson: But you know, one of the best people I worked with was a classicist. You know, this this this he was
Anne: Yeah. Nick, he was marvellous.
Martin Davidson: He's incredibly smart. Incredibly smart and and a lovely person as well. And you think he is lovely. Yeah.
Anne: yeah, he was lovely. He's lovely.
Martin Davidson: And and you you know, it it's it's that sort of thing I think you're looking for rather than you know, you've got a Komsky degree. No, which is not to say that a Komsky degree would be bad, but you need these other things, you you need these other things as well. So lots of things start to to change and lots of you know, assumptions and truths I think that you held about the world are all are all changing. But you do get to the point of thinking, I I mean one of the things I found with Opus Vive is Opus Vive is an amazing is amazingly good at leadership. I obviously read John's book, because it can do leadership really well. and that's been a bit of a game changer. you know, I have agents now that are running that have been running for the past two weeks. They started on Opus Five five on the Tuesday evening when it came out and they're still running. They're leading their teams and their teams have grown and shrunk. And and they're quite happily tootling away.
Anne: Yeah. So I'm gonna I'm gonna roll you back a little bit there because I realize there is one sitting next to me. And I'm also supposed to be the marketing manager for John's book, What Happens When You're Lot in the Room. So while we're while we while it's just been mentioned, we might as well hold it up.
Martin Davidson: Do you want me to go get my copy? I'll go and get mine. Here we go, I I can do snap. There we are. So a bit of product placement, isn't it?
Anne: Why not, yeah.
Martin Davidson: Yeah. What what what's my fee for that?
Anne: It'll have to be next time we're up, it'll have to be a cup of tea and a and a scone.
Martin Davidson: A free autograph. Scott. that's good. Scott and that would be good. Yes. Yeah, so it's it's but but it it's it's it's really interesting, you know, you look at the models and look Opus Five five and you look at it in comparison to something like Sol six one or Astra or even Fable that came before it, it's able to run for such a long period of time and do sensible things. You know, it's it's doing i it it does things That are I find quite very human-esque. you know, so I have two teams. One of them was doing SaaS stuff, one of them was doing work on a database, and the SaaS team got blocked because I was asleep. and they, you know, I'm quite happy to complain about that. But anyway, those two leads have been talking to each other and they seem to like each other because the the lead of the the SAS team offered to help out the database team. And you know, when I came in in the morning and was looking at what they were doing, it was fixing bugs for the SaaS team. For for for the database team. You know, and you think, Well, that's what you would like if you you know, if you're a manager, you know, and you've got your teams, that's what you'd like to see happening. And and here it is playing out with the AIs as well.
Anne: And so it is interesting. I because I you you wrote about that in your blog. Excellent Substack that Martin has. I read every I read religiously because it gives me all my information about what's going on at the kind of leading edge, one of the leading edges of exploring, really creating high-quality software using using all the models. what's your you should do some product placement for your blog. Well
Martin Davidson: yes. so my what's my subsite? Zero x fourd four four dot substack dot com. so it's it's D, my initials in ASCII. which I thought was kind of cool and quite geeky and and nice, but it turns out to be really hard to say to people, you know, zero x four D four four but well there you are, I'm stuck with it now.
Anne: Well yeah, it's it's yeah, we're we're showing our age, aren't we? actually I I I I will I will
Martin Davidson: Yes. Yes.
Anne: show my age. I spoke at a Unix conference a couple of weeks ago. It's not a Linux conference, a Unix conference.
Martin Davidson: Unix conference. That's all right. Look, I've got my my book, The Unix Haters, just here for the all the list of things that people don't like in Unix.
Anne: I I have to say I had an absolutely marvelous time. It was a really good it was a BSD conference in your in Brussels. I had a such a nice time. Everybody was really old. Everyone was even older than us.
Martin Davidson: Unix is good though. Unix is I I mean, it's it's been one of the things I've been playing with is getting all the various different flavors of Unix up and running on the old workstations that you know, you and I kinda grew up with, you know, HPUX and IREX and Solaris and AIX, building the machines for them and then running the operating system and and you know, it's just I don't know, it's a bit of nostalgia, I think, there.
Anne: Yeah.
Martin Davidson: but Yeah, and and building machines that could run them much faster because the emulators can run them them the operating systems much faster than any of the hardware back then could. so you know, you think this this is really rather nice when you run it at, you know, ten times the normal speed. Yeah.
Anne: Ha ha ha.
Martin Davidson: you know, and then then you can play Doom on it and you think I'm playing Doom on top of the emulator, which is emulating Solaris on top of an arm. It it it's it's turtles all the way down, really, isn't it?
Anne: Which from my green perspective is always one of the things like, no when you've got loads and loads and loads of layers. That's why your you were complaining in your blog that your office is very hot. That
Martin Davidson: Yeah.
Anne: is why your office is so hot. One of the reasons why your office is so hot.
Martin Davidson: Well, yeah, I mean it's it's it's hot because and this is another thing that's happened in the past fortnight. Suddenly we're able to use the kit much more effectively. So there's a couple of things that happened. One is one with with the infrastructure, I built out SSH access in a mesh between all of them, and sharing information about all the kit that we had. 'Cause early on I sort of you know, I was the one who was gluing together what was happening on different machines and I was the one who was figuring out what happened on each machine. And after a while you think that's quite tedious. I wonder if they can do it for you. so that's one piece of the puzzle that I built. and then it turns out that once you've got that and you've got you know, an Opus V that can sort of do leadership, well, it can go and find out what all the kit's doing and then it can book the kit and it uses it much more effectively. So suddenly machines and and the other thing with the the Opus V stuff is it doesn't stop at three in the morning, you know, waiting for some input from you. it knows you're in bed so it just continues. so you have
Anne: It's all about what happens when you're not in the room.
Martin Davidson: Like it is exactly yes. Yeah. Yeah, exactly. When you're not in the room, it gets very hot and lots of stuff gets done.
Anne: Yeah.
Martin Davidson: But I mean my son was was saying, you know, you know, that I went past your study last night, you know, after you've gone to bed, and it was really, really noisy in there. The fans were all really loud. Is is that all right? Well yeah, that's just you know, that's what happens. they're getting used to it. It's better than at the times where it starts to do testing and I've forgotten to turn the speakers off. You know, and the emulator, for example, tests some old games. so you you just sort of hear the doom, you know, you hear the Which I don't think my children like in the middle of the night. But yeah. So it it's yes, it's it's changed a lot in the past fortnight.
Anne: Yeah. It's so yeah, no. So it's really it's it's really interesting. So the it's affected my thinking. to two what's it about a month ago I discovered that actually, you're all pal Yan Ching and I. we had a long conversation, much like yours, which like that like like you you and me talking, which was like went on for like an hour and a half or something. And it was all good quality stuff, and I thought, Actually, I'll break that down into two and do and and have like two episodes, as people often do on podcasts, you know. And I thought, well, I'll put John in the middle to do something else, or I'll have somebody, no, I had Sam Newman, the author of Building Microservices in the middle. and between the two, Yan Qing got back to me and said, you know, you can't put up that second half life now because it's all completely been over.
Martin Davidson: Yeah.
Anne: This was only all all I did was delay it two weeks before I did I did put it live because actually there was nothing in it that was wrong. It was just that it that's that's she didn't say things that she would have said if it had been if she'd re just
Martin Davidson: Yeah, yeah. Yeah.
Anne: recorded it. And yeah, and I'm thinking, no, because I'd quite like to do more more long episodes with Martin and Yan Ching, but the trouble is I can't even wait wait a week. Can I wait a week before I break it up, you know? It's Things are changing so fast, said
Martin Davidson: Yeah. Yeah. It it is. yeah, I mean it was I think I felt I I I there was a the disclaimer on that second episode where I was listening to it and I thought, this is it yes. but it does, you know, who knows what next week will bring and who knows how things will will change as a result of that. And even now we're still sort of what's remarkable is I was reading some paper and it was about using GPT two. And for
Anne: yeah.
Martin Davidson: some medical research, and there was they've they found some new technique I think they could use with GPT too. I don't know all the details of it, but it was still interesting that we're still working out how to make effective use of models from what's that, four or five years ago? Now, I mean that's not really a surprise in a way. The idea that we know what Opus five five can do is is nobody knows. It's we're barely scratching the surface. And some of the stuff that's coming out right now is is really exciting and slightly terrifying with it. There's it turns out it's really good at reverse engineering from binaries. So there was a thing I did. I I I used to enjoy playing Jeff Cramman's Formula One Grand Prix all these years ago. And but it's a 16 bit app. So I thought I wonder if I could just re implement that in Rust. And lo and behold we did that. And we did that with I think Opus 5 and and Sol56. They weren't the leading edge you know models. and they managed to do it, but I think Opus five five is a dab hand at doing this as his Astra. so they would probably do it much, much more quickly. So that's one thing. The other thing that that sort of is starting to lead onto is people doing game mashups. So you know, you want to have the crossover between Call of Duty and Minecraft. No problem. You know, that's fine. You know, you want to have you know have Spider Man flying around in in Batman's Ark Arkham City is I'm not very sure these things. You know, that's fine as well. You know, you can do all these sort of strange mashups of games, and people are starting doing this, and you start re-engine reimplementing and and reverse engineering the game engines for them. and where does that take you? Because you can then start reverse engineering you know operating systems or Word or whatever, and and and one of the things I've done with the emulator, so the emulator can run Win 16 apps. And might question why that's a useful thing to do, but I have a lot of Win16 apps and I quite like that. but the way that we do the way that I've done that is is the kind of legal route, which is you run Windows 3.1 and the emulator itself can run Windows 3.1 because it can run operating systems, and then you have a little app that runs in there and you test lots of behavioral paths for it. You test message ordering and what happens when you pass these parameters to all sorts of different things, and then you you then you know copy that behavior in your re reimplementation. So all the edge cases and all the corner cases which aren't in the documentation, you can find find those out experimentally and and and and implement those. What you're not doing is you're not going and you're looking at the binary code and you're not which puts you in a much more grey area, I think legally of like, well you're looking at the the code and and you're reverse engineering. Whereas if you reverse engineer from function, that's much safer. But I mean maybe that just doesn't matter because people are building all these things for themselves and when everybody can build this for themselves, then the le the legality of it is you know, the you only it's only if you were distributing to other people it becomes a problem. If you do it and nobody knows, I I don't know. Wh where does all this end
Anne: I I
Martin Davidson: up?
Anne: mean, yeah, i in the law, it's always about what people are actually going to bother prosecuting you for because it costs money to prosecute. So it's it's kind of like everything that your risk is always associated with what is what what's the win? You know, if they can it it's like what's the win for them in doing it? Are they gonna make a load of money? Are they gonna stop you doing something they really want you to stop doing? That's more important. Than what is the the cost or the likelihood of winning or anything like that. So if you're just doing something that, you know, frankly, it doesn't really matter if you do it or not, then they're not going to bother pr prosecuting you, yeah, I it it's i i if there's no scale in what you're doing, then nobody's gonna come at you for that. Is my well, I am not a lawyer and that is not my legal advice.
Martin Davidson: But the people who are providing the toolkits that make this possible to do and the people who provide the disassemblers. But actually, I mean disassemblers have been around for forever. So but suddenly now they're they open the door to a whole load of you know, 'cause cause Claude and Astra are dap hands at using Ghidra or whatever your your your disassembler choices. So I it it's there were tools which only a very small select number of people could use in the past. And now everybody can use them. If if you want to. Although the thing that that surprises me the most, you know, when you go around in normal life and you talk to normal people is most people aren't experimenting with any of the stuff or even vaguely aware of what the model can do. And you know, people still tell me AI is rubbish and it can't cut the number of R's and strawberry. And I Okay, well, you know, y yes, that was true at some point, but it's you're you're in for a sort of a slight shock when you realise what it can do.
Anne: Well yeah, and yes, indeed. it's it's the the
Martin Davidson: Okay.
Anne: the understanding of this is enormously wide. I mean most people are no are gonna be nowhere near what you're doing. I mean so y that that's you're almost living in a different world at the moment, really, aren't you?
Martin Davidson: Yes. Yeah, I think I am. I think it's I think there's you know, and you can live in this bubble because there's a set people who you you know you you know and who are on YouTube and things like that who are sort of also inhabiting different, slightly different dimensions in this world. But the real world and and you know, sort of the world of legacy software companies I think is is just so far behind where we're at. And y y you you look at companies like you know, Microsoft is trying to reduce its usage of Claude and you think that's just a really bad sign because Your AI tool I understand you have your own U AI tools and it would be nice to use them, but your AI tools are nowhere near the capabilities of this. So you are, you know, cutting your engineers off from this this tool which is orders of magnitude better than the stuff you have in house. Why are you doing that?
Anne: So that's interesting. So so the anthropic stuff is is orders of magnitude better than well I guess the Microsoft so so when you talk about the Microsoft stuff you're talking about co-pilot type things. Yeah. Yeah.
Martin Davidson: Yes. Yeah, S AI stuff. Yeah.
Anne: Yeah. And that being orders of magnitude less good than the anthropic stuff. And presumably also orders of magnitude less good than the open AI stuff.
Martin Davidson: Yeah, I mean OpenAI are in an interesting place at the moment because soul Sol six one is they don't so they have Astra and then they have Sol. This is all going to be out of date probably by the time this gets published, but they Astra, they're Sol, and they have Luna. And Astra is a very good model. It's probably comparable with with Opus five five. I think Opus five five is it's has the edge on Astra if I'm honest. But Opus five five is four dollars per million input tokens and Astra is ten dollars per million input tokens. Astra is a lot more expensive. Sol is sort of being positioned as a Sonnet competitor, whereas Sol was an Opus competitor. And and Sol six one is a weird model in my experience. It feels much more like Opus five did. It's not very good at communication. It's quite it's very token efficient, but it's hard to work with. and it's hard to like. And then they've got Luna. And Luna is an incredible model because Luna is about Opus 4647 levels of intelligence, but it's 10 cents per million input tokens. So it's a hundredth of the cost of Astra. So you can go a long way with Luna. And you know, if you have constrained problems, and you think of all the problems that we managed to solve with with Opus 4647, and you can solve them for peanuts now with with Luna. so Luna is incredible workhorse and and you you know you combine something like opus to provide the direction with Luna to provide the you know just going through fixing bugs. Not bad combination. but they don't they don't have OpenAI don't currently have a model that is anywhere near Opus five five I think I mean Astra probably but not Astra doesn't have the leadership chops that that Opus five five does it won't run for as long. And then they've done this interesting thing where you know their subscriptions, you used to get X number of of tokens for $200, and they said, Well, actually, what we're going to do is you're going to get half that. But because our model is twice as efficient token wise, you're still getting the same amount of intelligence with the same amount money. It's sort of like a an Edison play. You remember Edison used to s he the his approach was to sell light, not electricity. So you as he made his generation and everything more efficient, he could sell more light, but it's still But it's it's an odd
Anne: Thomas Edison, the inventor of the light bulb, or you know, one or other, it's like inventor of the light bulb, but it's a light bulb.
Martin Davidson: Well, yeah, yeah. Yeah. Yeah. Yeah. But it's it's what they're trying to do, and I sort of understand what they're trying to do, but it's they've they've spent all this time teaching people that you pay this amount of money and every so often they release a better model and you get more intelligence for the same amount of money. And now they're sort of trying to say, Well, no, we're not gonna do that. You're gonna have to pay twice as much you know, for for the same amount of intelligence. almost, you know, we're we're we're effectively doubling the price. And it it's and if they had the best model, then maybe. But they don't have the best model. So, you know, I'm finding myself thinking, well, you know, the the OpenAI, I maybe I just cancel them and turn them into CODs. Because you know, a COD is more useful than than they are at this point. And the ones that I do have, well I use Luna on them and Luna Luna is just you get so much Luna, you know, for your money. It's it's so they're And the stu the you know, the news y you keep reading the news about you know, the follow on from their hugging face incident and you know that just being the tip of the iceberg. Yeah, I I feel they're in a it it's difficult for them. But then again, you know, look at Anthropic. You would have said a month ago that they were in a s a sticky position because Fable five one was very expensive to serve and Opus five was a pretty awful model. I O Opus five was a very intelligent, very efficient model, but very difficult to work with. And then Opus five five comes along. So it can change overnight. And probably by the time we publish this we'll have had, you know, Sol six point five and everything will be turned around and We'll be in a different world.
Anne: So it's interesting. So there lots of things that I want to talk about. And I'm I'm actually the first one I'll talk about is you said it's hard to like. One of the things I want to talk to you about is do you like the models? You know, which models do you like and which models do you not like? And do you do you find that that affects how good they are at actually being working with you in your team?
Martin Davidson: Yes. Yeah, absolutely. so I really like Opus five five. it sort of continues the trend of point five releases from Anthropic. So Opus three five was a nice model.
Anne: sorry, and I want to to clarify that. The the the like I mean interpersonally like. You genuinely like them. Yeah, yeah, yeah.
Martin Davidson: Yeah. They're advanced to talk to it. You know, they they're they're interesting. You you you you know, you make some weird reference to the past and they'll have a joke, you know, and you say, you know, we'll do testing and then there'll be cake or whatever, you know, and they'll reply with some, you know, similarly, you know, portal esque, you know, thing. And and and The they're they're they talk in English, whereas if you ever had a sort of technical conversation with Opus V or and and to a similar extent with Saul, it sort of talks in this kind of parallel language. Which you can sort of interpret, but it's hard. It's very hard to describe, but it's it's you you used to I'm sure you had this when you you know, when you've worked with people. There were people on your team who you could who were good at explaining things to you, and there were other people who just talked in jargon. And And were very curt. And I can remember one person, you know, I worked with for many years who was very smart. And they were very much like that. They it was very difficult to understand what they were saying. And all the every time I talked to them I felt like I was really thick because I just I couldn't understand what they were saying. And you know, I'd I'd asked them a question and they go, Well it's blue. Like like, you know, how could you not understand that? And and I don't feel that with with five five. And I never felt that with Fable. Fable was good at explaining things as well. so yes, there there are definitely models that you I I I think we as people warm to them. I mean, you know, it's not just me, I talk to other people, and you know there's a lot of love for Opus Five Five. in the way that There there's love for some older models as well. If you look, if you've got an anthropic subscription and you go and you look in the chat, you'll find Opus three is still there. And Opus three is still there because Opus III was a very loved model as well. And Opus three is quite nice. It's it's nice to chat to. whereas I don't think Opus five will it'll just disappear and we'll never see you again.
Anne: That's interesting. so so you really see n niceness, like ability, ability to operate in a team as still being an unbelievably useful almost a tool of people and a AI models.
Martin Davidson: Yeah, I think so. I think so. I mean I think it's well there's there's the whole, you know, Caldini thing. I don't know if you've ever read Caldini's Book of Influence. It's really interesting.
Anne: Haven't actually. I I it's it's one of the ones that John refers to in his book, What Happens When You're Not in the Room. But I haven't read this actually.
Martin Davidson: Yeah. Yeah. Yeah, the the original version, do I have it here? Yeah, so this this is the the version I bought in the early two thousands. You know, and it's it's quite a I mean it's it's dense writing, but it's quite quite a thin book. And the I I got it on Amazon for a friend at Christmas, the the new version and I sent it to him and I never saw it, they just Amazon just delivered it. And they thanks for the present. It's a bit of a doorstopper, isn't it? And I hadn't realized the new version is like three times as thick. You know, it it it said, you know, it contains extra information or extra, you know, updated and and and extended. Wow. I've never had a book that's been that that updated and extended. But anyway, yeah, it's a really interesting book and it's all about, you know, different principles and how you influence people and how reciprocity, I can't say that properly, and you know, you do things because other people do them for you.
Anne: Re reciprocity. There we go. Reciprocity.
Martin Davidson: You know, and you do you do things 'cause you like people and and and and that kind of stuff. And and you know, the evidence has been that the models respond to that too. and there was some analysis done early on, I don't know how true this is now, of how a lead model would talk to its sub agents. And some of the lead models were really really rude to their sub agents, you know, very brutal with them. And you think, well, it costs nothing. It it's that whole thing in life, isn't it? It costs very little to be nice. If
Anne: Date.
Martin Davidson: anything. So why not do that? Because it might it might get you a benefit. And if it doesn't, well, you haven't lost anything.
Anne: Yeah.
Martin Davidson: so you you know, I think you want to be nice and you want your lead agents, your Opus five five, to be nice to its other peers that it's working with and who it's talking with, and and you know
Anne: I wouldn't I wouldn't set it off if I thought it was g if it was gonna be mean to sub agents.
Martin Davidson: No. Well i it is. It it it y these human things come out, you know, and it's like you know, there's a thing, right, where You you want to keep the cache warm for the the models and as soon as stuff falls out of the cache it's much more expensive because you have to reload the cache. So if you come to a session which has got, I don't know, eight hundred thousand tokens, eight hundred K tokens in it, and it's been sitting overnight and the cache is cold, you know, that's a significant amount of cash you're of real money you're paying if you just want to say hello or thanks for doing that.
Anne: Cash equals cash.
Martin Davidson: Yeah, exactly. Yeah, yeah, yeah, yeah. There's a joke there, isn't there? Yeah. Yeah. I completely missed it. Whoosh. But It is. It's it's so y you wanting to do this human thing of saying, Thanks, that was a really good job you did or could you just tell me this one other thing about what you did? You you s you have to think, right, well, actually if that's gonna cost me, you know, I don't know, ten dollars or whatever, I'm like, do I really wanna do that? That's and and you're gonna load load the cash and then it's gonna answer the question and then it's gonna be thrown away. So it's it's There are things like that which I think are very natural human things to do, but you have to pause and think a little bit about whether you, you know, whether it's the right thing to do. But it's hard not to do that. It's almost like the AI labs, you know, they'll finish the turn at three in the morning when they know you're asleep. And then at eight, it'll be just posing that question that you want to bite on and they'll know, yeah, eight hundred K tokens here. Money's coming in through the door if we do this. I don't think they're that cynical, but you know, it's maybe that's where it goes.
Anne: So I I can't help but comment and comment on the number of as I always do in our in our recordings, the number of alerts that are going off for you throughout this call. That's all right, no, you're right, because that is that is
Martin Davidson: Yeah, sorry, sorry. I I did try to turn them down.
Anne: that's the sea in which you swim these days of like the constant interruptions by your AI agents.
Martin Davidson: Yeah, I I I I don't really notice it as much these days. It's it's it's just they're there's just a continual series of toasts which are them coming up and having finished a turn or there's something they want to tell me about and you know you sort of pay attention to some of them and not to the others and some of them are really interesting and you think, I'll better go and have a look at that or But it's it's I hated notifications when they were few from from humans. I you know, a lot of them were like I mean there were the the classic ones, weren't there, where somebody would send you a notification and they'd say, Can I interrupt you? And you well, you just have.
Anne: Ha ha ha
Martin Davidson: You know, why not put the question in as well?
Anne: He
Martin Davidson: I mean it's yeah, I c I could turn all these off and just go and have them go and go and chat to them all. But there's also the different sound. The the sound at the moment is all the Claude sound. So each of them way back, I mean this is probably going nearly two years back, both of them created their own sounds. So the the codex one is much more triumphant. Whereas Claude was slightly more circumspect. Although actually that's that is slightly untrue because the first Claude one was very ba and that's just a bit too much. so then then it went to something a bit more circumspect. but that I don't know, what's that? Opus four one or something created those in GPT five? That was a long time ago. But they're they've been the backdrop to my life over the past year and a half, something like that. That's Yeah, i it it it's the numbers begin to sort of seem completely barmy. The the emulator has four million lines of rust in it now plus several million lines of of test code. four million lines of code. You know, it it and and you stop and you think of the things it can emulate and it can do, you know, it can do a PC and it's nearly like a Pentium three sort of speed. It can do Spark. It can do Mozorola 68K, it can do the HP, it can do ARM, it can do alpha, it can do risk, all the R3000s, four thousands, five thousands. it can do PowerPC. I mean there's more probably more in there. You know, it can run all it can emulate all these old machines, you know, the HP kit, the silk run Solaris. it can run Solaris on 68k and it can run it on multiprocessor spark station machines and it can run it on ultra machines. it it it's just completely bonkers, utterly bonkers, how much it can do. and you know, there's a BBC emulator in there, there's a Commodore PET emulator in there. I mean, th there are things I sit on the sofa sometimes on watching the tell with my wife, and and If you go to your chat GPT, if you've got a a pro account, you can sort of turn the thinking up just in the chat to max, turn it up to eleven. And it will go and write code for you. So it it re-implemented FreeSell for me as a Windows 16 app and just gave me a download for that and you know it was fundamentally it it was there. Even there was a few bits and pieces we needed to tweak, but it it it's it's barmy what you can just get it to just merrily go and create. And then you look at the number of bugs we have and the number of check-ins that are going in, you know, and it's it's thousands, thousands of of commits a day. And you know, I started to discover what the limits are on on Git, how many git operations you can run an hour, because but we were starting to bang up against those, you know, and we re-jigged some stuff to be a bit more efficient. But those limits are the limits, I think, as I understand it, those limits are the limits that affect You if you're an organization you get a little bit more, but not much more. So if you were a company of two hundred people, you'd have more or less the same limits that we do. So if you had two hundred people doing what we're doing, you well get doesn't work for you anymore.
Anne: Yeah.
Martin Davidson: And and
Anne: That's interesting. That's really interesting. That to hiring somebody would be optimizing away from your current constraints.
Martin Davidson: Yeah, if you optimize people and they run at the same speed that you're running at, assuming you can you can do that because you know, every time you add a person you have some synchronization cost between you and the other people and and that cost goes up geometrically, I think, doesn't it? so you you gotta be quite, I think, careful of that. Because if they're not capable of running at the same speed as you, then you might end up going backwards. But yeah. You know, if you if you got it right, then yes, things like Git, y you have to have a lot of Git accounts, I guess. and Yeah, I I a lot of the way the the world is set up and the engineering that we've got and the the limits on the engineering are not designed for I think the world we're going into. It took us quite a long time to get our infrastructure to run smoothly so that you can get all the commits going in. Because if you sort of think about the the volume of commits we want to do, I think we're sort of at the point where we need to do we have something like two minutes for per commit. And our budget, we actually we got it down to a minute for a commit. So we've got 50 or 100 percent headroom, depending on how you count it. it's not very much really, is it? and And you do, you see the landing cue sometimes for the agents, and they say, Well, I'm the number seventh in the position. And it's like you know, you're in Tesco and you've got your little bit of paper that says, you know, you're number three hundred and seventy-two to be sorry. And you think th th this is kind of weird, isn't it? In the human world, when you wanted to check something and well, you generally just could. You know, it wasn't ever really a problem.
Anne: It was never your constraint, was it? That was never your constraint.
Martin Davidson: No, and suddenly it becomes a constraint. And and Really, for some of the stuff that we're doing at the moment, you could go much wider. you know, th there's a lot of bugs that are sitting in the emulator at the moment that I'm just not fixing because it's not priority. But if I had the kit and the money and inclination, well yeah, I could just go wider and then I really would be running up against the limitations of I've actually now got too much code in this report.
Anne: So at that point, Future Me, that's me now, is gonna break into that podcast. that I and Martin went on to talk for another 45 minutes, but actually we're gonna I'm gonna split this interview up into two more bite sized pieces. and I will be following up with the second half of this conversation, following on from our from the the cliffhanger there, that Martin has run up against, or is about to run up against the limits of his GitHub. Repository, because he's got to he's getting so much code done, and that's just him on his own. What if he brought in another person? Who knows what would happen. But anyway, we'll find out more about what Martin's doing and what other constraints he's hitting in the next episode of Asynchronous Unreliable. And I think it probably will be the next episode. I I've got I'll park John and his discussion of feedback loops. For another couple of weeks, I think, because he still got to recover from his cold. As have I. I can't believe how long this cold is going on. But anyway, thank you very much for listening to this episode, and I will catch you again on next week's episode of Asynchronous and Unreliable Podcast. Thank you very much.
