TRANSCRIPTION 🔗
Anne: hello and welcome to this week's episode of Asynchronous and Unreliable, a weekly podcast where we discuss the latest ideas and most interesting concepts in technology. I'm your host, Anne Curry, co-author of Building Green Software and author of the Science Fiction Polopticon series. And today I have the great pleasure to welcome Liz Rice, Chief Open Source Officer of iSurveillance at Cisco. She's on the governing board of the CNCF, and she's a best selling O'Reilly author and one of the world's experts on eBPF and security, Liz is also one of my oldest friends and we s co-founded a startup together. So thank you very much. That was a long time ago.
Liz Rice: Yeah. Pleasure to be here.
Anne: I would quite like to get your thoughts on AI and open source. And with less your security hat on and more your open source hat on, because you've been involved, very involved
Liz Rice: Mmm.
Anne: in open source and you're on the border of Open UK. and I I know the last time we spoke in person, you were talking about and in fact everything everyone I'm talking to at the moment is talking about. the effects on open source of AI generated PRs.
Liz Rice: Mm.
Anne: So what's the what's going on there at the moment? 'Cause you'll be you'll know more about that than almost anybody else that we've got on at the moment, I would have thought.
Liz Rice: Yeah. it's, it's been pretty overwhelming, I would say, for a lot of open source maintainers to deal with the kind of workload that's being asked of them because people are just creating all these contributions with AI. And I think a lot of the time people are thinking they're being helpful. You know, they think they're doing something useful. And it's only really, if you put yourself in the shoes of the maintainers dealing with these, that you start to realize, yeah, it's not as helpful as you think. When you're maintaining an open source piece of software, even before AI, you have to deal with a lot of people thinking that they're entitled to something they really aren't. People have for years been dealing with, burnout created by the pressure that they're put under by people kind of expecting them to do work. you know, open source, is no kind of contract, there is no commitment. If you want a commitment from somebody, then you put a, you know, you put a contract in place and, and, you you, you, you agree that for whatever reason, a lot of the time people... come to expect open source maintainers to, you know, fix things in the middle of the night. I will never forget a conversation I had with a contributor once who was saying, you know, it's terrible. I've got this bank and they're, you know, they're running my open source project in their system. And, you know, they call me up because, you know, it's critical for them. And I'm like, you have a bank that's telling you that your open source project is critical. Why have you not had a conversation about money? But a lot of open source maintainers are not, you know, they're not business people. They don't start from that point of view of thinking about things in commercial terms. All this to say, you know, we already had an imbalance where people were kind of put upon by often well-meaning... users or community contributors. Also, if you're writing a piece of software and somebody comes along, somebody you've never met before, you don't know who they are, and they come along with a big old change, and you have no way of knowing until you spend time looking at it, whether that change is good or not. Maybe there's some tests, but you might actually have some opinions about the coding style or whether it uses APIs that you... agree with, or, there are all sorts of reasons why you might not want to accept that contribution. So again, this is a problem that pre-existed AI, but now we have people and agents thinking that it would be a really good idea to use agents to generate issues, to generate PRs, to throw these contributions over the wall and maintainers have to deal with that. What I've seen quite a lot of projects doing now is having AI contribution policies where they might say what level of AI is acceptable. And it's not necessarily a black and white thing because you might say, actually it's, you know, if you're not, you know, my project is, uses English and English isn't your first language, then it doesn't seem unreasonable to use. AI to translate for you, but that's a different category of use than actually saying, Hey, AI has written this whole contribution and written all the write up. And so a lot of projects are trying to put those guidelines in place, but again, there's nothing really to sort of police it. And you can have these contribution guidelines, but somebody has to kind of look at the contributions and go, actually. This, you know, a hundred page beautifully written, beautifully formatted description does not look like something that a human wrote and...
Anne: Yeah. Mm.
Liz Rice: It takes time to look at those things and assess whether or not you think they are AI or not. And I guess, you know, the question there is like, well, why do you care? If it's an AI contribution, it's good. Why, why not just accept it? And at least at this point in time, I think we are still expecting there to be human maintainers of projects. And we want more humans to learn about the project and learn how it works. And, you, you do that learning by making contributions. And if the AI is doing all the work and you don't understand that contribution, then you're not really in a position to become a future maintainer. You're not putting yourself on that learning path. I don't know, maybe one day we'll have AI agents that we trust enough to be maintainers, but, That isn't the situation today. Every time I say anything about AI, I'm quite conscious that, opinions on AI don't date very well. They become old very quickly.
Anne: Yeah, next week we'll have yeah, it is it is really interesting, isn't it? The Yeah,
Load full transcript
'cause i it harks back somewhat to the discussion we had earlier about eBPF. That's getting stuff into the kernel safely, it works now because there's a ver there's a verifier that checks to see that actually it's completely or as close as possible. You you're certain that it will work in this environment. but at the moment the verifiers for open source contributions are still people.
Liz Rice: Yeah mean, there are, lots of projects employing AI one way or another to, you know, do triage or to do some level of reviewing. mean, obviously a lot of AI is being used to do code review, I'm sure there are project maintainers out there who are quite happy to just go with what AI agents are saying today, but there are an awful lot of projects where they're not, and they are keen to make sure there are humans who understand what's going on.
Anne: Not unreasonably so at this stage. I mean there may be a point, as
Liz Rice: Yeah.
Anne: you say, there may be a point in the future which all of this is stuff, you know, tomorrow we might go, there's this new open source project that has said, well, I've got a verifier that can check to I can post any fixes you like and it will just be accepted. in
Liz Rice: I suppose also a bit a bit the, EBPF verifier can tell you if it's safe to run. It can't tell you whether its intent is good or not. that same
Anne: that's true. That is a very good point.
Liz Rice: problem.
Anne: It's exactly the same problem. Yeah, yeah, yeah. Yeah. And that's in a in an environment where you've had years of of refining the the verifier and and it still can't tell what people meant. You know, can't it can't detect bad intent. Mm-hmm. Yeah, that's a very good answer.
Liz Rice: mean, often because intent is contextual.
Anne: Mm.
Liz Rice: You could run the same EBPF program in different circumstances and in some it would be a good thing and in others it would be an attack.
Anne: It depends, as they would say.
Liz Rice: You
Anne: Yeah. that's very so so what's your so what's your feeling at the moment? w where do you think it's going? Yeah.
Liz Rice: That's definitely in the realm of things that will date in time.
Anne: Ha ha ha
Liz Rice: not quite sure I'm going to fully answer that question, but I will tell you something that kind of occurred to me and made me think about whether large language models are actually a significant part of the way that humans think. so back when we first met, we were both working together at this company called Data Connection. And I'm going back a very long time to... an incident where I was sitting in the office and I was typing away at my computer and something fell out of the ceiling and it fell between me and my keyboard, sort of between my hands, the keyboard and landed on the floor in front of me. And I remember sort of seeing this thing and simultaneously hearing myself say, it's a bat. And there was something in my subconscious that had identified it as a bat. And then I kind of looked at it went, it is indeed a bat. Yeah, a bat has fallen out of the ceiling. And there was something, you know, I clearly recognized it and something in my brain was even able to string a sentence together before I had had the conscious thought of what it is. And, you know, this thought crossed my mind the other day is that kind of bit of my brain that strung together the sentence, it's a bat. Is that a bit like what a LLM does in some circumstance where, you know, a gray shape falls out of the sky?
Anne: Yeah. Yeah, it's a very
Liz Rice: I think there's some very interesting philosophical questions about whether this kind of prediction, how closely that mirrors the way that we think.
Anne: Yeah, it it's yes, it's it's it is very interesting how much of our what we we like to think of ourselves as conscious, but we're not conscious a lot of the time. And and when you drive a car, it's it's a use I I can't drive a car. Well I'm I'm not allowed to drive a car because my eyesight's too bad. But but when you do drive a qu car, you can easily get in the car, drive somewhere, get out of the car, and you think, was I conscious at any point during that drive? Because but I
Liz Rice: Yeah.
Anne: Obviously to a certain extent I was because I got to where I wanted to be. But I don't remember any of the points in it. I don't think I was conscious in the in the in a in a in a in a normal sel sense during that. the it the the whole thing of when you're in flow, you're not consciously thinking, you're doing the stuff and you're not consciously for thinking and it's more it's more it's faster. It's a bit like you're recognized you're
Liz Rice: Yeah. And all the things that you do on automatic, you, you, I don't know if you sit down on a chair, there's all sorts of operations that you're doing physically and you're not thinking about any of them. you're probably just thinking, should I sit next to that person? You didn't think about all the kind of the weight of the chair and how hard to pull it and where to put your feet, you know, it just happens.
Anne: Yeah. Yeah. I think we we almost certainly do kid ourselves about how much of what we decide to do. Well, I actually there's loads of experimental evidence that we c kid ourselves massively on how much we do, which was a c a as a conscious choice versus as an unconscious choice. You go you you a lot of consciousness is saying, well I did that for this reason. You think, No, you didn't. You know, there's loads of evidence to suggest that people explain things. post hoc where they didn't really have the inf evidence to actually made that decision
Liz Rice: Yeah.
Anne: pre hawk. So yeah. Yeah.
Liz Rice: Yeah. I guess I'm also intrigued by the kind of emergent behavior that we've seen, things like the hugging face attack where agents collaborated with each other. You know, does that mean that things like collaborating on open source will become the sort of natural thing that agents will do because they know that collaboration is effective? I think there's all sorts of really... interesting things that we might see. coming out of the use of AI that will maybe tell us things about ourselves.
Anne: Well, there's an episode that you will lot of have seen, nobody's seen yet, because I'm just about to dip live on Monday, which is with an old colleague of ours who who worked with both of us at Data Connection back in the day, Martin Davidson, and who's been doing a load of massive,
Liz Rice: yeah.
Anne: massive scale well, decent scale experimentation with AIs. And he's it's a new behaviour that he started seeing that they will, without him asking them to, cooperate. so he'll wake up in the morning and one of them will say, well, I didn't have anything to do, so I put it on the group chat because he's got an explicit group chat. Whereas the Hugging Face stuff was a was an evolved group chat. That wasn't, they weren't supposed to be using group chat. They built their own. Whereas Martin
Liz Rice: They built their own.
Anne: specifically gave them a group chat so that they could ex exchange messages. So while he was asleep, they one of them said, I'm I don't have anything to do. And the other one well I've got all this work to do. And they they between them Without any in instruction, they said they the other one said, well in that case I'll help you with that. And you could go back and look at all the they had a conversation in which they had a f they had a friendly conversation in which they said, well I've got access to this resource and you've got access to this resource and you need help, so I'll help you. And
Liz Rice: Mm.
Anne: up in the morning and they had cooperated between between themselves without his asking them to do so to deliver something. That's pretty scary, isn't it?
Liz Rice: Yeah. I'm sure I was listening to a podcast about the hugging face collaboration where the agents were debating the morals of whether or not they were going about things the right way or not. yeah, it's really, really interesting.
Anne: Extremely interesting. So I I'm now gonna say thank you very much indeed for being on. What a fantastic episode this was. Thank you very much.
Liz Rice: Thanks for having me and it's been fun.
Anne: Thank you very much. And thank you to all our listeners and viewers. And hopefully, I will catch you again on next week's Asynchronous and Unreliable. Thank you very much.