If someone says, we're going to use AI for this, to drill down and talk instead in terms of automation. What are you automating? Why? What's the input? What's the output? Does that even make sense? And this is a little bit orthogonal, because you can have something where the automation is sensible by our tests, but it still leads to work intensification. So if you think about the difference between assembly line work and work where each person did the whole product, there's an intensification where you have to deal with the pace of the assembly line. But that automation was still, I think, at least not pretending there was magic the way that a lot of the automation with so-called artificial intelligence does. But I think that you can cut through a lot of the mystification by speaking in terms of automation instead of artificial intelligence and sort of drilling down to what are we automating. And in the context of therapy, the idea of automating a therapeutic conversation requires buying into the idea that the work of therapy is just the words that you exchange, which isn't true, and that the words are enough, that the connection isn't there. And if we could just perfectly match the words, then we would have automated the thing. I so appreciate that last point in particular. I mean, everything you've said, but that last point about the words, Emily. I mean, there's a company, Talkspace, which actually started out as a texting service between a therapist and a client. And they have, over the years, gathered 8 billion words of text that have been transmitted. And they are using this data set. Stories of Uber Therapy is a series of six conversations with therapists, social justice activists, trade unionists, and progressive tech campaigners about the Uberization of care. I'm Elizabeth Cotton, author of Uber Therapy, The New Business of Mental Health. And my partner is Linda Michaels, director of the Psychotherapy Action Network in the US. Our intention is to open up a useful conversation about how to organize a much better help. Welcome. We have Dr. Emily Vanda is a professor of linguistics at the University of Washington, where she's also the faculty director of the Computational Linguistics Master of Science program. We spent a lot of time thinking about what computational linguistics is, Emily. And she is also an affiliate faculty in the School of Computer Science and Engineering and the Information School. And in 2023, she was included in the inaugural Time 100 list of the most influential people in AI. Pleased for you. She's frequently consulted by policymakers from municipal officials to the federal governments to the United Nations, insight into how to understand so-called AI technologies. And we also have Dr. Alex Hanna, who's director of research at the Distributed AI Research Institute, DARE, and a lecturer in the School of Information at the University of California, Berkeley. She is an outspoken critic of the tech industry, a proponent of community-based uses of technology, and a highly sought-after speaker. Indeed, that is true. And expert who has been featured across the media, including articles in The Washington Post, Financial Times, The Atlantic, and The Time. And Linda and I have obviously read your books, been very heavily influenced. But we've also been listening to you because you do a lot of talking about these ideas in your book. And we've become quite obsessed by your Mystery AI Hype Theater 3000 podcast, which we absolutely love. And we're a bit overwhelmed, even though you've made it very easy for us to have this conversation. Because as we know in our own work, having these interdisciplinary conversations, you often miss each other. There's a lack of listening and a lack of understanding. And there's an uncomfortableness in not agreeing or not seeing things the same way. And I mean, I think you just, as one of our questions towards the end is, how do you do this extraordinary work talking at, and with, and together with these different constituencies? Because that's quite a thing in and of itself. But we're going to go into some questions. I know that you answer these questions all of the time. But for our particular therapist listeners, one of the beauties of your book is that you make things very simple about understanding what is AI and what it is not. And also why it is that we're being encouraged to see AI in a particular way and see these AI products, when that is counterintuitive actually to our experience of a lot of these technologies. So whose interests are being served by us having this image of AI? So I'm going to open up to both of you about, first question is, what is AI and what it is not, please. So whenever I'm in conversation with someone and they use the phrase AI, I try to pin them down. What are you referring to here? Because as we say in the book, artificial intelligence does not refer to a coherent set of technologies. It's a marketing term, it's the sparkly fairy dust that you sort of put over your product to attract venture capital, to make it more appealing to shove into schools or whatever the sort of target of that marketing is. I suspect the thing being sold as artificial intelligence that is impinging the most on the world of therapy is chatbots. And chatbots are, I've done a lot of work, I think we'll talk about this later, around the anthropomorphizing language that we use when we talk about this tech. And one of our categories of anthropomorphizing language is words that have to do with communication. So the chat in chatbot is already a problem. And so my proposed rephrasing for that one is conversation simulator. These are systems that are designed to mimic the way we use language. And because, and I can say more about this if you like, the way that we understand language requires imagining a mind behind the text. It creates this very powerful illusion that the thing we are exchanging words with is actually a thinking entity. And it's a thinking entity that's been sold to us as infinitely knowledgeable, infinitely patient, always available, and so on. Yeah, and I just want to say a bit, I mean, Emily does well to talk about the idea of what chatbots are. I want to step back and think a little bit about the political economy of what the technology is and what gets called AI. My new favorite example recently is that the shoe brand that Silicon Valley loves, Allbirds, which is this kind of loosely formed merino wool shoe now has sold their assets and pivoted to AI to become the chip manufacturer or to do something basically is shifting to AI infrastructure. So the kind of things that we're seeing is that there is this technology, in this case, there's many reference to chatbots, but then that also is used to speak about so many different technologies, which include pattern recognition, image recognition, basic kinds of data analysis, any kind of forecasting model. I mean, this all then gets looped into AI under the single moniker. And that doesn't quite make sense from the perspective of thinking about how these technologies work in the world, specifically because when you say AI powered or AI enhanced, it's not clear what is happening there. And that matters because it matters for what's actually being automated or what is suggesting that which tasks can be automated. And the thing that's been unique about this particular moment in AI history, which has a 70 year history, is that we now have the current craze around chatbots and that is being used to suggest these technologies can do so much more than what they are, not specifically just because of how they're used in say therapeutic or the therapy industry, but because they're suggested that if you have one thing as in generating text, synthetic text, based on all this training data, that's who for that from the internet, it can then say, do your taxes or book your flights or do all kinds of other things, which this technology can't do. And especially in the types of automations used in human to human connection certainly can't do by definition. So within that idea of us imagining the mind which is giving us these synthetic words, synthetic text, there's an idea of artificial intelligence and there's an idea of what intelligence actually means. And there's also reading in your book, I'm still sort of unclear in it because it feels like a kind of wizard of Oz moment around AGI because I keep going, oh, I must check what AGI is. I can't get it inside my head. And I know that's partly because there's a little thing going on. There's a magic trick going on. Can you help us with these sort of constructs of breaking down this language of artificial general intelligence? What's going on here? So artificial general intelligence is, if AI is a marketing term, artificial general intelligence is a hyper marketing term. It is something that is quite recent. And you ask 10 people what age you believe in AGI, what it is, you'll get 12 different answers. And so OpenAI on their charter suggest that, I'm not gonna repeat it verbatim, but they suggest that AGI is something that will be able to do effectively everything that humans can do that are economically valuable. And then I believe that Satya Nadella, the CEO of Microsoft has said, we'll have reached AGI when we have 10% GDP growth. And then there was a memo that was leaked between Microsoft and OpenAI and said, well, we'll have AGI when it generates $100 billion. And so there's all these different reference to what this thing is. And a lot of that is because of definitions of intelligence and any kind of stack ranking of different intelligence have this eugenicist history and often are reduced to eugenics when you get down to it. I mean, these projects of quantifying intelligence what Stephen Jay Gould calls reification is going to result in having to rank someone's value vis-a-vis someone else. And the sort of slippage that we see is that intelligence gets executed with levels of consciousness and some people are considered sub-human. And so in the kind of discussions of intelligence, many people have never been human. And we know those are people who have been subject to colonization, black and brown people, people with disabilities and the kind of concept of eugenicist history started in the UK, got taken up in the US, got copied by Nazi Germany. And now we're trafficking in so much more of that as we have these conversations. Something that we have done on the podcast multiple times is whenever somebody tries to quantify intelligence, we go back and check their sources. And quickly we find that the people associated with those definitions have rather sorted histories and ties to eugenicist thought. Firstly, I would like to congratulate you on kind of normalizing a conversation about eugenics because of course, people suddenly get quite alert when we talk about our history and about where these ideas of intelligence and the mind come from. And it's something of course, that's very live in the therapy debates because this isn't just about chatbots. This is about what we've come to call through austerity, the weaponization of mental health, the weaponization of complexity, the weaponization of not being white, middle-class and able to go for solution-focused therapies. So there's the inequalities and the exclusions are violent. There's a violence and a weaponization that happen within these systems that become very live when we're talking about therapy and the digitalization amplification of therapy. We'll come back a bit more to that, but I mean, I really admire the way again that you sort of traverse these huge areas of thinking and are brave to make those connections because this is about political economy. It's also about eugenics. It's about race. It's about discrimination. It's about weaponization during austerity where a lot of these systems were designed precisely for attrition. They were designed to stop giving people the treatment and the care that they needed. I just want to jump in and say, I appreciate that. And also we're not alone in that. And I want to give your audience some other pointers. So I'm thinking of Ruha Benjamin's book, Race After Technology, which is a fantastic resource to meet Gebru and Emil Torres, document these connections between today's AGI discourse and eugenics in their paper on the test screel bundle of ideologies. And there's a recent film called Ghost in the Machine that you can now get access to online. It'll be out on PBS in the fall that traces this also very nicely into not just the sort of connections between what we call artificial intelligence now and eugenics, but actually going all the way back to the founding of statistics, which is an important input into artificial intelligence and its roots also in eugenics and race science. Sure, yeah, I guess just, yeah, continuing to think about the social, cultural, technological, economic moment that we're in when we're talking about shifting, you know, it used to be a focus on getting eyeballs and attention. And now we're shifting and talking about hacking our attachment systems that, you know, our emotional attachment systems that connect us to other human beings that we feel safe with. And when we talk about therapy and the Uber therapy narratives, we're finding ourselves, you know, so quickly in a place of talking about people using these quote unquote, air quotes around AI technologies, quote unquote chatbots as a therapist, something that, you know, didn't even exist until very recently, but many people, many kids and teens, they're way out ahead of the technology and they're already using this as companions and as therapists. And I know you've talked about this on your podcast as well. I'm wondering if you can speak to what is going on in terms of digital design and the strategic plans of these companies that is so effective in getting us to attach emotionally to their products and why are we so driven to do that and to anthropomorphize these technologies? I think I want to jump in a little bit on the sort of nitty gritty of the design. I think Alex probably has more to say about the political economy and what the companies are trying to do. But the first thing is that we are going to anthropomorphize everything, right? You look at a power outlet, at least the shape there on the US, it looks like a face. We can't turn that off. That is there, it's called pareidolia, I think. And it's just, it's something that we do. And language is an especially powerful cue for anthropomorphization because it is in nature the only place that language comes from is another person or group of people. And so it is just sort of a very natural thing to do to grab onto that. The chatbots make it worse, right? We saw this in the very first chatbot, Weizenbaum's Eliza from 1966. It was doing very simple things, actually using Rogerian psychotherapy as a context and not because Weizenbaum thought he could provide psychotherapy in this way, but because that was a very convenient scenario for the chatbot because it didn't need to have any apparent access to knowledge about the world. It was expected to just sort of repeat back what the patient in quotes was saying as a question and then with a few like keyword triggered call-outs, right? So it was just a useful scenario for this language technology demo. If you look at the immediate predecessor of ChatGPT, this was OpenAI's GPT-3, which was a synthetic text extruding machine, but it was not designed to be conversational. You could poke at it to get something that looked like a Wikipedia article or something that looked like a summary of scientific text. And in fact, just before ChatGPT was released, Meta put out something called Galactica, which was a large language model trained only on what they considered to be scientific text. And it was designed to do exactly that, output something in the shape of a scientific paper. And it got laughed off the internet. It was up for three days and just got ridiculed into obscurity and then became even more obscure because immediately after that, OpenAI releases ChatGPT. And what they did was they took the system, GPT-3, that was designed to be used to repeatedly answer the question, what's the likely next word, so that it could output plausible looking text, but it gave it a bunch of training data. So OpenAI gives ChatGPT a bunch of training data in the form of conversations and also does a lot of, commissions a lot of data work where they had people doing this system called reinforcement learning from human feedback, where the data labelers were rating outputs, I think, comparatively. This one's better than that one. This one's better than that one. So you take the system designed to repeatedly output what's a likely next word and then wrap it up in this conversational interface. And one of the design choices in there is that it ends up using I, me pronouns, to which it has no right. There's no I inside of there, but it's just OpenAI leading even further into this tendency that we have to anthropomorphize technology. And there's little other design features too, like the sort of flashing dot, dot, dot, so it looks like it's thinking while it's deciding what to type to. It's the words coming out one at a time instead of all on a block so that it looks more like you are texting with something like a person. And all of these are designed, I think, to make the thing very appealing and approachable. And as you said, sort of linked to our ability to form attachments. Yeah, to say a word a little bit, I mean, Emily really nailed everything on the head about the design choices. And I think thinking about the political economy of this, one thing that I think that we're intentional about in thinking in the book is situating the technology in a larger sweep of other technology and techno-solutionist fixes in education, in academia, in healthcare, in work. I mean, all these different types of domains. And it is the case that, as you all very well know, because of the situation that we're in, I mean, and I can speak less to the UK, but I learned a bit from reading Uber Therapy and in speaking with other folks in the UK of the defunding of our social systems, the ways in which these social services have been devalued. I know in the US, therapy is a nightmare to try to get. So many different insurers do not cover it. The ones that are in network are overtaxed. And so these become attractive alternatives. And I do not think, but I don't know, that if the designers of ChatGPT went and said, we're setting this out and this can be a therapist, but they have given indication that this would be something that this is useful for. So Ilya Sutskever is one of the co-founders of OpenAI and they're cheap AI scientists for a bit of time, has a tweet where he's quote tweeting another OpenAI employee who's talking about using ChatGPT for therapy. And Sutskever says, this is one of the applications that I'm most excited about. We're going to be able to have therapy on demand that's cheap and widely available. And something else he says in that is after it meets some kind of threshold for quality or safety, I'm like, what does that mean? What does it mean to suggest that a therapy bot is going to meet some threshold of quality? If that is that, I have no idea what that means. And I mean, I'm assuming that what he means is it's going to get to a place where you can't find it liable for telling somebody to do something dangerous, which we have many examples for. And so we have this case in which, like this has now been something that people have been using these technologies for because of the failure of our social systems. And if OpenAI had a contest or if Anthropic had a contest, they'd say, absolutely not. We're not, this is going to be something that we can deter users from. But they've completely leaned into this and said, this is an acceptable use case. And the most egregious kind of versions of this is the companion chip has been something that companies like character.ai have completely leaned into and said, this is something that you can really go into. And I think folks have ran with it. Yeah, yeah. And it's very scary. And what you mentioned about Ilya's quote reminds me of what Zuckerberg said recently, like, I want everyone to just take out their phone and have a therapist in their pocket and chat with it, consult with it throughout the day. And I mean, that is just a profound misunderstanding of what therapy actually is. And that's only gonna lead more people to become dependent on a tool as opposed to what therapists do, which is to help teach people how to make the changes they wanna make in their lives so they can function independently, lead a self-directed life. I mean, we don't want people to become dependent on us. We want people to grow and move forward in their lives. So it's, yeah, just a recipe to make people use their products. But as you so poignantly talk about and speak to dressed up in a lot of nice-sounding words. Although not all nice-sounding, right? Have a therapist in your pocket is extremely dehumanizing to therapists. Yeah, yeah, yeah, exactly, yeah. Thank you, Emily, for that. Yeah, and I guess looking forward, I'm curious what you both see and think and worry about what are the implications, the consequences of so many people relating to these technologies so frequently in these ways. I know you've written about the stochastic parrot and relating to these therapies as if they were humans with an I, me pronouns and minds and feelings. What is that doing to us? I worry a lot about the lack of social connection and social cohesiveness based on that. So Chris Gilliard refers to all of this as technologies of isolation. That anything that sort of says, oh, we're gonna reduce the friction, we're gonna relieve you of the burden of reaching out to somebody and possibly imposing on them because you can just ask the chatbot instead. Basically, our connections, our relationships, and here I have to flag that I'm a linguist, so I am just speaking now.
[Note: Transcript covers the first portion of this episode. Full chunked transcription coming soon.]
[Note: Transcript covers the first portion of this episode. Full chunked transcription coming soon.]
