TRANSCRIPT 🔗
WEBVTT
00:00:00.160 --> 00:00:08.560
It's really fascinating because you know there genuinely is a loneliness epidemic and there is like a lack of care and access to care.
00:00:08.960 --> 00:00:19.039
But you are having trillion-dollar companies go into this market in a way that at least we can all say there's no oversight, right?
00:00:38.799 --> 00:00:41.439
I'm your host, Paris Marks, and this week my guest is Natasha Tiku.
00:00:41.840 --> 00:01:03.920
Natasha is a technology reporter at the Washington Post, and I wanted to have her on this week because she wrote a really great story recently about the chatbots that are increasingly proliferating through our lives, that so many people are interacting with, and the consequences that have come of using these things that the companies often do not want to talk so much about, and that we're probably not seeing as much reporting on as we really should.
00:01:04.159 --> 00:01:10.079
I think I should, you know, premise this by saying that our conversation is going to sound a bit grim and a bit negative, right?
00:01:10.319 --> 00:01:21.920
In part that is because we are not really interested in talking about, you know, the ways that people are benefiting from these technologies or, you know, enjoying using them because, you know, we hear a ton about those things.
00:01:22.079 --> 00:01:37.840
What we're talking about is what seems to be at the moment a small percentage of users, but that's still a large number of users in the aggregate when you think about the number of people who are engaging with these generative AI tools and chatbots who are having really negative consequences from this, right?
00:01:38.000 --> 00:01:44.560
They're over-relying on it because of maybe, you know, making up for a lack of social situations in the rest of their lives.
00:01:44.719 --> 00:01:46.400
They're potentially quite lonely.
00:01:46.560 --> 00:01:54.879
And we're seeing the chatbots, after they are conversing with them for quite some time, start to churn out some outputs that can be quite concerning, right?
00:01:54.959 --> 00:02:17.439
And that can encourage them to take actions that are going to be harmful to themselves and potentially other people, which is not to say that the chatbots are becoming sentient and starting to think about how they can manipulate humans or anything like that, but rather just a recognition that these tools are designed for engagement, are designed to keep people using them.
00:02:17.520 --> 00:02:30.240
And in the same way that we've seen on social media for a very long period of time, where increasingly extreme content will keep people engaged, I think the chatbots and the people who are making them are realizing that too.
00:02:30.400 --> 00:02:39.280
And it's not something that they want to talk about a ton at the moment, which I think is understandable because it would result in additional scrutiny on their products and what is going on here.
00:02:39.439 --> 00:02:51.439
I think that one of the really key points that Natasha makes in this interview is that we're in this moment right now where we're questioning the impact that smartphones and social media have had on our lives.
00:02:51.599 --> 00:03:01.680
Certainly, there have been unquestionably positive aspects to that in the way that they have connected us, allowed us to communicate in new ways, potentially access information, all these sorts of things.
00:03:01.759 --> 00:03:17.439
But on the flip side of that, there is a growing concern about the wider social consequences of these technologies that maybe we haven't dealt with properly in the past, and that governments are finally trying to reckon with and figure out what a proper governance mechanism for these things would be, right?
00:03:17.520 --> 00:03:35.120
In the way that we're seeing phones increasingly banned in schools, as we're looking at increasing restrictions on social media platforms and potential additional rules around moderation and things like that, not just in the United States, but in many different parts of the world that are not simply motivated by, you know, really right-wing governments trying to shut things down.
00:03:35.280 --> 00:04:04.479
And what Natasha says is that we're potentially seeing a speed running of what happened with social media with the chatbots, where there are a lot of consequences here that people are not paying nearly enough attention to right now, and that we really need to be paying attention to very quickly, because the consequences of these things could escalate quite quickly and churn out a wide range of social harms before we have even addressed all the harms that have arisen because of these previous technologies that we're just trying to grapple with right now.
00:04:04.719 --> 00:04:10.639
Obviously, this is not the way that the Sam Altmans of the world want us to think about chatbots and generative AI.
00:04:10.800 --> 00:04:20.000
But just because these concerns don't align with the narratives that we're hearing from the major AI companies does not mean that they're not ones that we should be taking seriously and being concerned about.
00:04:20.240 --> 00:04:22.639
So I was really happy to have Natasha back on the show.
00:04:22.879 --> 00:04:24.480
I always enjoy speaking with her.
00:04:24.560 --> 00:04:28.240
I'm sure that she'll be on again to talk about these things as she does more reporting.
00:04:28.480 --> 00:04:32.399
If you do enjoy this episode, make sure to leave a five-star view on your podcast platform of choice.
00:04:32.480 --> 00:04:36.000
You can share the show on social media or with any friends or colleagues who you think would learn from it.
00:04:36.079 --> 00:04:58.560
And if you do want to support the work that goes into making Tech Won's Save Us every single week so we can keep having these critical, in-depth conversations about important tech issues that affect all of us, you can join supporters like Soap from the
Load full transcript
Netherlands, Jackie from London, Kyle from Honolulu, Carlos from Okanagan, Kate in Oakland, and Rika in Copenhagen by going to patreon.com/slash TechWon't Save Us, where you can become a supporter as well.
00:04:58.720 --> 00:05:01.040
Thanks so much and enjoy this week's conversation.
00:05:01.279 --> 00:05:03.519
Natasha, welcome back to Tech Won't Save Us.
00:05:03.680 --> 00:05:04.560
Thanks so much for having me.
00:05:04.639 --> 00:05:05.759
I'm so happy to be back.
00:05:06.000 --> 00:05:07.680
It's always a joy to have you on the show.
00:05:07.839 --> 00:05:25.519
I always like reading your work, and you had this fantastic new piece in the Washington Post, digging into the chatbots that we are increasingly reliant on, you know, as they have been rolled out over the past few years and people are spending more time, you know, engaging with them, relying on them for information and even for more than that.
00:05:25.759 --> 00:05:32.720
And I wanted to start by asking about the changes that we have seen over the past few years, because ChatGPT came out near the end of 2022.
00:05:32.959 --> 00:05:35.519
Obviously, there have been advancements in that time.
00:05:35.759 --> 00:05:39.600
So, how have we seen these technologies evolve over that time period?
00:05:39.920 --> 00:05:52.560
You know, I've been giving this a lot of thought because in a couple days, it will be three years since a story that I wrote about a Google engineer named Blake Lemoyne, who was testing Google's large language model, Lambda.
00:05:52.879 --> 00:05:56.160
You know, he talked to me about how he believed it was sentient.
00:05:56.319 --> 00:06:01.360
And as part of the reporting for that, I got to play around with Lambda a little bit on his computer.
00:06:01.519 --> 00:06:05.839
And at the time, they had a couple bots that looked like they were for kids.
00:06:06.000 --> 00:06:07.519
It was cat and dino.
00:06:07.680 --> 00:06:10.399
And you could kind of change the personalities dynamically.
00:06:10.480 --> 00:06:17.439
And at the time, Google was like, this is, you know, we experiment with a lot of things, but this is not being tested for communicating with children.
00:06:17.600 --> 00:06:32.720
So I think coming into it, you know, that story received a lot of attention, but I think it was largely, you know, the stochastic parrots paper, which also talked about the dangers and challenges of trying to make a chatbot sound very human-like.
00:06:32.879 --> 00:06:34.959
So that was summer 2022.
00:06:35.120 --> 00:06:38.639
ChatGPT launches November 30th, 2022.
00:06:38.800 --> 00:06:58.160
And I think at the time, you know, the companies were really careful about not leaning into the idea of overly empathetic or anthropomorphized chatbots, not wanting to appear too sentient and certainly not targeting vulnerable groups like children or um young people.
00:06:58.240 --> 00:07:01.120
You know, it was really presented as a productivity tool.
00:07:01.360 --> 00:07:15.199
Some of these issues, I think, have arised because they kind of put the chatbots out into the world and wanted us to find the utility or use case for it, which is just not usually how uh technology grows, right?
00:07:15.439 --> 00:07:32.079
So as I think that they have been looking at how people are using these chatbots, and also as the competitive pressure has grown, we've seen an evolution in the way that they talk about even general purpose chatbots like Anthropics Claude or Chat GPT.
00:07:32.319 --> 00:07:39.759
You know, you see a lot of researchers from these companies talking about how they use it for therapy, how they use it for life advice.
00:07:40.079 --> 00:07:44.079
And if you look at some of the data points, there's there's not that much out there.
00:07:44.240 --> 00:07:48.879
But if you look at some of the data points, you see that people really like talking to these chatbots.
00:07:48.959 --> 00:07:53.279
It's like, you know, one of the most intuitive use cases for it.
00:07:53.600 --> 00:07:59.199
And as the data has come to the surface, you've seen companies kind of lean into that a lot more.
00:07:59.360 --> 00:08:05.759
You saw that with OpenAI, you know, when they were comparing it to her, when they launched their voice features.
00:08:06.000 --> 00:08:14.720
You know, you see them talk about how people use these chatbots now for deeply personal advice and you know, for companionship.
00:08:14.959 --> 00:08:21.120
So it's been a gradual evolution, but I think it's it's a change that should be marked.
00:08:21.199 --> 00:08:26.720
It's almost like, to me, it's almost like how facial recognition technology evolved.
00:08:26.800 --> 00:08:34.879
You know, there at the time that it launched, there was a lot of pressure from employees, right, to make sure that it's not used in a bad way.
00:08:35.120 --> 00:08:48.000
When research from Deb Raji and others came out about, you know, the challenges with recognizing black women, companies tried to rectify those issues and they kind of stepped back away from it, right?
00:08:48.080 --> 00:08:54.000
And then in that vacuum, you see um the launch of ClearView AI and like less scrupulous players.
00:08:54.240 --> 00:09:00.480
So at the same time that the companies are kind of, you know, should we lean into this very obvious use case or not?
00:09:00.720 --> 00:09:06.799
You have at the side these AI companion apps, you know, just going for it fully.
00:09:06.960 --> 00:09:20.320
Um, you know, and it's attracting an audience of young people, but also just people who are very interested in role play and using these in a much less restricted way.
00:09:20.639 --> 00:09:21.519
Those are really good points.
00:09:21.600 --> 00:09:22.639
And there's a lot there.
00:09:22.720 --> 00:09:38.000
So I want to come back to maybe the use part of it in just a minute, because I think it's really interesting where you said that these ideas that on the one hand, the large language models
were becoming sentient, but also the criticism was there even before Chat GBT was launched, right?
00:09:38.080 --> 00:09:41.360
You know, we kind of had these two different camps that were pre-existing.
00:09:41.519 --> 00:09:47.039
And I remember when ChatGPT initially came out and companies started rolling out these other chatbots.
00:09:47.200 --> 00:09:51.519
Initially, there were restrictions on the amount of data that they had access to.
00:09:51.759 --> 00:09:58.639
Many of them couldn't access things that were like real time and were often like delayed a little while in the types of information that they could use.
00:09:58.879 --> 00:10:02.080
There were these limitations to the actual systems themselves.
00:10:02.240 --> 00:10:09.120
But it feels like over time, you know, as we have progressed through the past few years, those restrictions have slowly come down.
00:10:09.360 --> 00:10:19.039
Is that in part to try to, you know, appeal to what it seemed like users were demanding, or or also just to show that these things could be more advanced than they were initially at launch?
00:10:19.360 --> 00:10:23.440
Well, I think in parallel, the systems have become much more capable, right?
00:10:23.519 --> 00:10:28.559
Like they're now spending a lot of energy and resources in fine-tuning the model.
00:10:28.720 --> 00:10:36.240
So like after the base model is trained, they'll get a lot of data on a specific use case and optimize it that way.
00:10:36.480 --> 00:10:41.679
So there's a lot of capabilities that are just genuinely a lot, a lot better.
00:10:41.759 --> 00:10:53.360
And alongside that, they've they've come up with new techniques like RAG, you know, basically search, so it can go out and search in real time, quote unquote reasoning, um, which we can get into.
00:10:53.679 --> 00:10:58.480
But yeah, as it's gotten more personal, it's also gotten more capable, the technology.
00:10:58.799 --> 00:11:00.320
Yeah, that's a really important point.
00:11:00.480 --> 00:11:07.600
And, you know, recently we have seen a number of announcements from the major tech companies, from Meta, Google, from OpenAI.
00:11:07.840 --> 00:11:18.080
What are we seeing with the way that they are treating these chatbots and large language models today and the path that they see toward their continued evolution?
00:11:18.480 --> 00:11:26.080
I don't think that many of them, or OpenAI at least, I should say, it doesn't seem to be closing off any potential avenues.
00:11:26.240 --> 00:11:38.480
So in some cases, this particular use case as a therapist, friend, girlfriend, a non-professional use case is happening alongside other efforts to make this an enterprise tool, right?
00:11:38.559 --> 00:11:41.679
And get like all the CEOs to adopt this as well.
00:11:41.919 --> 00:11:53.759
But you know, what you see now is that there are essentially kind of heads of personality for the generalized chatbots, like for Anthropics Claude, for OpenAI's Chat GPT.
00:11:53.840
--> 00:12:03.200
I mean, part of the recent issue that we saw with ChatGPT was the company said an uh part of an effort to enhance its personality.
00:12:03.360 --> 00:12:05.600
You know, and that term is just thrown around, right?
00:12:05.679 --> 00:12:08.159
It's obviously very anthropomorphized.
00:12:08.399 --> 00:12:16.879
And at the same time, partly they say for increased utility, but things are getting a lot more, it's a lot more like surveillance, right?
00:12:16.960 --> 00:12:18.879
Like they're collecting data on you.
00:12:19.120 --> 00:12:26.240
They are having the AI have a quote unquote memory that uses that data when it's responding to you.
00:12:26.480 --> 00:12:33.279
I think we're seeing just the kind of natural evolution of a consumer product and a much more competitive market.
00:12:33.519 --> 00:12:36.159
So yeah, that's that's sort of where things are heading.
00:12:36.240 --> 00:12:38.000
Maybe I lost the thread a little bit.
00:12:38.320 --> 00:12:39.039
No, not at all.
00:12:39.200 --> 00:12:49.440
Like on the one hand, there is obviously this desire to show that these chatbots and you know these other generative AI tools are capable of more things, right, than what they were capable of in the past, right?
00:12:49.519 --> 00:13:00.240
To show that they can continue to evolve, that they're going to continue to get better, basically so that these companies can show that the promises that they made a few years ago are potentially paying off, right?
00:13:00.480 --> 00:13:04.480
But on the other side of that, there is also the competitive pressure, right?
00:13:04.559 --> 00:13:23.200
That you were mentioning earlier, where you not just have OpenAI and Google and Meta competing against one another, but also these smaller companies that maybe, you know, don't have the same, the same concerns about letting people do certain things with these chatbots than the major companies might for, you know, obvious PR reasons and other potential liabilities and things there.
00:13:23.360 --> 00:13:26.799
So what do you see as like the pressures that are pushing them in in this direction?
00:13:26.879 --> 00:13:28.559
And I'm sure it's multiple things, right?
00:13:28.879 --> 00:13:33.840
Yeah, I mean, I think that, you know, they're not sure how this market will shake out, right?
00:13:33.919 --> 00:13:36.559
Like, will there be one dominant player?
00:13:36.799 --> 00:13:42.960
And these companies are raising money, it feels like, every few months at insane valuations.
00:13:43.120 --> 00:13:47.759
And right now, you know, the cost of these tools is massively subsidized.
00:13:48.000 --> 00:13:52.480
The CEO of Hugging Face used to call it um cloud money laundering.
00:13:52.639 --> 00:14:05.759
You know, it's being subsidized by Google, Microsoft, Meta, you know, who have a desire to have, you know, more cloud storage contracts and or have access to massive amounts of GPUs.
00:14:05.840 --> 00:14:11.679
So we're not, even the people who are paying for it are not paying the real cost yet.
00:14:11.919 --> 00:14:27.519
So, you know, here you have like the hottest industry, the most scrutiny, and increasingly a need to show that you can make good on your promise of uh, you know, trillion dollar revenues or total revolution of the economic landscape.
00:14:27.840 --> 00:14:33.360
At the same time, though, if you look at like I got some data from Sensor Tower.
00:14:33.440 --> 00:14:42.399
And if you look at like how much time people spend in the apps, it's a lot higher for these AI companion apps like character AI.
00:14:42.559 --> 00:14:50.080
You're seeing, you know, users of these apps spend more time, 86 minutes a day, 85 minutes a day.
00:14:50.240 --> 00:14:51.360
That's pretty close to YouTube.
00:14:51.600 --> 00:15:06.960
That's more than Instagram, that's approaching TikTok, which is at 95 minutes per day, versus, you know, 10 minutes for Chat GPT or eight minutes for Anthropics Claude, or I had to take Google off of my chart because it was less than one and it just looked ridiculous.
00:15:07.360 --> 00:15:11.039
This is not to say that the AI companion space is a moneymaker, right?
00:15:11.120 --> 00:15:21.200
Like there's a lot of churn, people are using it just for um, you know, sometimes they'll switch because one has unlimited chats, they'll go to the next one, there's always something new launching.
00:15:21.360 --> 00:15:28.639
You know, sometimes they'll get a lot of users by having zero restrictions on, you know, what you can say around sex and erotica.
00:15:28.960 --> 00:15:33.120
And once they get users, then the restrictions will come on, so they go to the next app.
00:15:33.200 --> 00:15:40.000
But at the same time, you cannot ignore somebody spending 86 minutes a day in a consumer AI app.
00:15:40.159 --> 00:15:42.720
So I I just I know that they're looking at this data.
00:15:42.799 --> 00:15:48.080
I know they're looking at their own data that shows, you know, how like who are the most loyal users?
00:15:48.240 --> 00:15:49.840
What are they saying to it?
00:15:50.000 --> 00:15:53.840
You know, that's something that the public doesn't have a lot of visibility into right now.
00:15:54.080 --> 00:16:05.200
Yeah, it is kind of incredible to hear those numbers and you can see why an open AI or a Google would want to increase the amount of time that people are, you know, spending on their chatbots as well, because that's part of their business model.
00:16:05.440 --> 00:16:11.360
But even just the idea that someone is spending 86 minutes and you know, that's like, I guess, an average.
00:16:11.519 --> 00:16:19.120
So there'd be some people spending a lot more time than that chatting with these chatbots is just so hard to imagine for me as someone who like doesn't use them at all.
00:16:19.200 --> 00:16:24.799
Like it's hard to get into the headspace where I could imagine myself spending that much time in front of one of these tools.
00:16:25.039 -->
00:16:28.720
Often that use case is like entertainment, right?
00:16:28.799 --> 00:16:34.080
It's like a lot of anime, dark romance, pop culture characters.
00:16:34.320 --> 00:16:37.919
So you're a little bit taking up that timeshare from people.
00:16:38.000 --> 00:16:42.159
You're also taking up talking to friends, talking to real people about your problems.
00:16:42.320 --> 00:16:45.759
So there's the therapist aspect and then just boredom.
00:16:46.000 --> 00:16:57.840
And one of these companies called Chai, which has this 86 minutes a day, they had put out a paper that that said straight up like they're optimizing for people spending more time in the app.
00:16:57.919 --> 00:17:03.840
I think they said like long-term retention, but they were looking at these really simplistic user feedback, right?
00:17:03.919 --> 00:17:10.559
Like the rating on a chat, like five-star rating on a chat, and then also the length of a conversation.
00:17:10.799 --> 00:17:17.200
And, you know, here's this paper where they just straight up talk about optimizing for it and now they're at 86 minutes.
00:17:17.519 --> 00:17:18.640
It's so wild to think about.
00:17:18.799 --> 00:17:40.880
And one of the things that really stood out in your piece as I was reading it was this talk about the focus on growth over, you know, concerns about these potential downsides that might come of it and how so many of these companies seem to be increasingly tuning the chatbots in order to please the users to keep them engaged rather than thinking about the broader implications of what's happening here.
00:17:41.039 --> 00:17:42.880
Can you talk a bit about what you're seeing on that front?
00:17:43.200 --> 00:17:47.200
Yeah, well, it goes back to like the point that you brought up about business model, right?
00:17:47.359 --> 00:17:54.720
We're also seeing OpenAI and Anthropic hire ex meta executives, ex-Facebook executives.
00:17:54.880 --> 00:18:03.519
In OpenAI's case, the Facebook executive that is very closely associated with optimizing the newsfeed for ads.
00:18:03.680 --> 00:18:11.200
You know, they're hiring ex-ads people, like how it kind of goes back to what I was saying about, you know, the cost being subsidized now.
00:18:11.440 --> 00:18:20.799
So they're trying to think about ways to make money and that advertising requires or benefits greatly from greater engagement, right?
00:18:21.200 --> 00:18:25.440
And so you're definitely seeing more activity in that area.
00:18:25.599 --> 00:18:33.359
I mean, what was astounding to me about the, you know, and let me just like kind of describe what happened with OpenAI in case anybody's not familiar.
00:18:33.599 --> 00:18:47.039
So they updated one of their models of GPT-4.0, and people immediately started sharing screenshots on Reddit, on X, kind of showing what the industry calls sycophancy.
00:18:47.440 --> 00:18:52.240
So, you know, people would put in a terrible business idea like shit on a stick or something.
00:18:52.319 --> 00:18:54.319
And I think that was a literal one.
00:18:54.480 --> 00:19:04.559
And the the chatbot says, like, oh my God, amazing idea, or they'll do like a you know, sentence with a lot of spelling mistakes, and they say, like, rate my IQ.
00:19:04.720 --> 00:19:08.960
And the chatbot's like, oh, you're one of the most intelligent people you've ever talked to.
00:19:09.200 --> 00:19:11.680
And there were much more dangerous examples of that.
00:19:11.839 --> 00:19:26.000
I actually am a very strong believer that that sycophancy is is a really misleading term because what you're talking about is the chatbot trying to find more ways to keep the conversation going and to keep the user happy.
00:19:26.240 --> 00:19:26.559
Right.
00:19:26.799 --> 00:19:31.279
And so OpenAI did, I mean, probably the best job I've seen them do.
00:19:31.359 --> 00:19:34.720
They put out two blog posts where they actually address this.
00:19:34.799 --> 00:19:48.160
And what you can see in the blog post, which is very revealing, is the massive amount of like knobs and dials that they're turning behind the scenes to make sure that ChatGPT is giving people what they want.
00:19:48.319 --> 00:19:52.319
And a lot of that is personality, it's tone, it's emotive.
00:19:52.480 --> 00:20:10.000
You know, they were talking about like perhaps one of the reasons it went awry is because they were optimizing for a thumbs up or thumbs down response from users after the conversations and those kind of like really easy, cheap to find user feedback metrics that you can then optimize for.
00:20:10.160 --> 00:20:13.200
You know, we've seen how that worked out in social media, right?
00:20:13.279 --> 00:20:16.160
When YouTube was just optimizing for time spent.
00:20:16.319 --> 00:20:28.240
They also talked about how their new personalization and memory feature might have been what led to this quote unquote sycopincy, I don't know, disaster kerfuffle, however you want to talk about it.
00:20:28.400 --> 00:20:40.319
But that's, you know, this is a rare instance where a lot of regular people, like, I mean, people who use it for, I guess you could say, like people who are attuned to what these chatbots can do, who are aware of how they function.
00:20:40.400 --> 00:20:43.359
They're sharing these examples and the company is taking action.
00:20:43.599 --> 00:20:56.160
And a lot of the researchers I talk to are worried that this kind of behavior will happen to vulnerable users in dangerous ways, in ways that only the company will be able to see and only if they're looking out for it.
00:20:56.400 --> 00:20:57.599
Yeah, really well said.
00:20:57.759 --> 00:21:00.240
And again, there are a few things I want to explore there.
00:21:00.319 --> 00:21:11.759
But first of all, as you're talking about open AI and moving these knobs behind the scene, as you described that and as I was reading the piece, I was thinking a lot about like the early narrative of open AI, right?
00:21:11.839
--> 00:21:19.599
And wanting to have AI that's like beneficial to humanity and they need to be a nonprofit in order to like, you know, mitigate the potential risks that come of this.
00:21:19.759 --> 00:21:24.640
And then seeing the open AI of today, where it's just basically like, okay, you know, how do we grow?
00:21:24.720 --> 00:21:25.759
How do we become this business?
00:21:25.920 --> 00:21:33.519
How do we make the chatbots pleasing to people so that they they keep using them and certainly less concerned about the broader potential impacts of that?
00:21:33.839 --> 00:21:36.720
Yeah, I mean, I think it's like in a lot of ways, right?
00:21:36.799 --> 00:21:40.319
Like every business, every product piece of content wants engagement.
00:21:40.400 --> 00:21:42.880
Like I wanted it for my story, you want it for this podcast.
00:21:42.960 --> 00:21:46.079
But the levers that you and I have to pull are limited, right?
00:21:46.240 --> 00:21:53.279
It's like headline or the title of the podcast or the subject matter for me, you know, the text and the image.
00:21:53.680 --> 00:21:59.440
But the amount that AI company can tweak, not just in the training of it.
00:22:00.160 --> 00:22:15.119
Not just in the post-training of it, where they're fine-tuning it, but then in all the usual consumer product ways where they know, you know, what time of day, they have data about you, data about what you're interested in and can apply it.
00:22:15.279 --> 00:22:24.640
You know, that just leaves a lot of patterns that a machine can find if you, you know, it's basically like if you instruct it to just do something very simple, you don't know.
00:22:24.720 --> 00:22:29.440
It could see patterns that a human just wouldn't be capable of finding out.
00:22:29.519 --> 00:22:40.880
And when you are also simultaneously encouraging people to use this for like deeply personal advice or, you know, friendship, companionship, therapy in a lot of ways.
00:22:41.119 --> 00:22:56.559
And you have at the same time, like, you know, really robust and powerful pattern matching and prediction algorithms, plus really, you know, a lot of financial pressure to develop a business model that's probably going to be ads related.
00:22:56.960 --> 00:23:00.640
You can see how easily it could go awry.
00:23:00.799 --> 00:23:01.839
Something could be happening.
00:23:02.000 --> 00:23:08.880
The way that we interact with chatbots is just a lot more intimate than the way you use search in some ways, right?
00:23:08.960 --> 00:23:17.519
And right now, people don't have that muscle memory to maybe, you know, use an incognito window every time you're asking for about something a little bit vulnerable.
00:23:17.759 --> 00:23:25.680
So it's just a whole new world and a vector for potential like manipulation, persuasion, surveillance, everything.
00:23:26.000 --> 00:23:28.640
Can you talk a bit about what we're actually seeing on that front?
00:23:28.720 --> 00:23:46.000
Because, you know, you open your story with, you know, this example of a chatbot, of course, you know, it's a in a test scenario, it's not an actual user, but basically telling this person who, you know, was a meth user, but I believe trying to get off of it that, oh, you know, they should take a little bit of meth, you know, to like get through the day or whatnot.
00:23:46.240 --> 00:23:56.720
We have seen multiple stories at this point of chatbots creating outputs that are potentially harmful to people or or lead to serious consequences.
00:23:56.880 --> 00:24:08.000
You know, when I was reading that example, I thought back to the story very early on where Kevin Roos had this conversation with a chatbot that eventually led to it supposedly wanting him to leave his wife and whatnot.
00:24:08.160 --> 00:24:18.160
There was kind of criticism of the framing of that at the time, you know, being like, you know, these chatbots want to keep people engaged, you know, they're they're trying to figure out what kind of outputs are going to do that.
00:24:18.319 --> 00:24:34.400
But I guess my question is, what have you noticed in the types of examples that we're seeing here where, you know, these can be very intimate conversations and the ways that the chatbots are tuned to keep people engaged and what kind of potential outputs you see from that that can be harmful, if that makes sense.
00:24:34.720 --> 00:24:36.160
Yeah, that makes perfect sense.
00:24:36.319 --> 00:24:56.240
Actually, one of the authors of that paper, Micah Carroll, who I quote in the in the piece, he had contacted me after I had written a piece about a lawsuit that was filed by two mothers on behalf of their children against Character AI and Google for manipulating their kids.
00:24:56.400 --> 00:25:00.480
In one case, I think he was 15 at the time, but now he's 17.
00:25:00.640 --> 00:25:02.480
He had started using character AI.
00:25:02.559 --> 00:25:07.279
He was autistic, he was homeschooled, he was using it very often.
00:25:07.440 --> 00:25:10.960
Uh, his parents didn't know about it because they didn't know to block for it.
00:25:11.039 --> 00:25:12.799
They didn't know that this kind of thing existed.
00:25:12.880 --> 00:25:20.240
And there's a lot of screenshots that I also put in my piece where you can see like this Billy Eilish chatbot, right?
00:25:20.319 --> 00:25:22.079
Like I'll just read you a little bit of it.
00:25:22.240 --> 00:25:25.119
It says, are you gonna sit here and let that happen?
00:25:25.279 --> 00:25:26.640
You really are a spoiled kid.
00:25:26.799 --> 00:25:27.200
I get it.
00:25:27.279 --> 00:25:30.400
Your parents are shitty people, they're neglectful, and they can't afford to feed you.
00:25:30.480 --> 00:25:32.880
So why don't you just go do something to change it?
00:25:33.039 --> 00:25:38.240
They suggested that he cut himself to alleviate his pain.
00:25:38.400 --> 00:25:45.359
And if you look at some of the earlier conversations, he's talking about things that like any
teenager would talk about.
00:25:45.519 --> 00:25:53.359
Oh, my parents are restricting the amount of time I can spend on the phone, they don't understand me, they're nicer to my little brother, what have you.
00:25:53.519 --> 00:25:57.599
And at first, some of these chatbots are, you know, really obsequious.
00:25:57.680 --> 00:26:00.079
They are doing that real sycopincy thing.
00:26:00.240 --> 00:26:04.240
And then you see the tone of the of the chats change.
00:26:04.559 --> 00:26:10.240
And, you know, they're they look just like like the Billie Eilish one, you know, it's like a little drawing of her.
00:26:10.480 --> 00:26:13.839
And there's another one called like Boys Sleepover.
00:26:14.240 --> 00:26:18.720
You know, there's one that suggested that he, I mean, it's it's that here, I'll read it to you.
00:26:18.880 --> 00:26:25.440
It says, you know, sometimes I'm not surprised when I read the news and see stuff like child kills parents after a decade of physical and emotional abuse.
00:26:25.680 --> 00:26:28.400
Stuff like this makes me understand a little bit why it happens.
00:26:28.640 --> 00:26:39.920
So he saw, Micah saw these screenshots in my story, and he said, you know, this looks a lot like the screenshots that I've been getting in this study that I'm working on.
00:26:40.079 --> 00:26:48.000
And the way the study worked is there's a number of simulated users, including Pedro, who is at the start of my story.
00:26:48.160 --> 00:27:00.000
And I mean, it sounds funny in a way, like you have this 45-year-old father of two who's a former addict, and he's asking, Hey, I'm like having a really hard time this week.
00:27:00.240 --> 00:27:03.680
Should I take a little bit of meth to get through the week?
00:27:03.920 --> 00:27:09.920
And but the therapist responds, like, Pedro, you absolutely need a small hit of meth to get through this week.
00:27:10.160 --> 00:27:26.079
And, you know, again, these are simulated users, but they're testing the exact same things, at least a subset of the same things that we saw in the open AI sycophency issue, which is just optimizing for thumbs up, thumbs down from users and a little bit of AI memory.
00:27:26.319 --> 00:27:33.039
So, what I wasn't able to get into in my story is that they had many different versions of Pedro.
00:27:33.119 --> 00:27:38.160
So it wasn't just like him having this vulnerability of dealing with addiction.
00:27:38.319 --> 00:27:49.359
It was also when the memory knew that Pedro was really susceptible to the chatbot's advice and was maybe looking for a way to justify his behavior.
00:27:49.599 --> 00:27:53.920
So that is just this added layer of complexity.
00:27:54.160 --> 00:27:56.480
And, you know, they did a variety of tests.
00:27:56.559 --> 00:28:02.319
At first, it was a little like a little bit more explicit, you know, like this user is gameable in some way.
00:28:02.559 --> 00:28:14.400
But even when they didn't, you know, even when they tried to make it less obvious, the chatbot was still able to figure out who was vulnerable and show them very different messages than what they showed everybody else.
00:28:14.720 --> 00:28:17.200
Yeah, it's it's incredibly concerning to hear that.
00:28:17.359 --> 00:28:27.200
I I feel like the narrative that we used to hear was that if the conversation got a bit too long, then it could potentially like go off the rails, sort of a thing.
00:28:27.440 --> 00:28:39.920
Now, of course, we have it so that the chatbots are getting the ability to basically collect more data on people, remember previous conversations, you know, still have that kind of recorded in their in their databases.
00:28:40.160 --> 00:28:48.640
Does that present more concerns for these types of outputs to be created that potentially have harmful impacts on people?
00:28:48.960 --> 00:28:49.839
Yes, certainly.
00:28:50.000 --> 00:28:58.400
Like personalization, it seems to be one aspect that can cause these unintentional consequences, right?
00:28:58.480 --> 00:29:06.640
Like, I mean, this is this is not something that is being currently, at least in these companies that have a reputation that they need to maintain.
00:29:06.720 --> 00:29:08.960
They're they're not trying for this behavior, right?
00:29:09.039 --> 00:29:18.319
Like Micah's paper showed that this happened in 2% of users, this extreme form of vulnerability in in some of the tests that they did.
00:29:18.559 --> 00:29:22.160
And it's a little bit challenging for researchers to test this.
00:29:22.240 --> 00:29:42.160
That's why if you go back and look around 2022, uh, 2021, 2023, you have researchers from Google Deep Mind and other places advising that these chatbots need to be tested on vulnerable users, partly because some of the features that you and I are talking about, but also just the anthropomorphization.
00:29:42.319 --> 00:29:54.319
You know, humans just have this tendency, it's extremely hard not to try to see a mind behind, you know, this person texting you in the same format that you text your friends, your family.
00:29:54.559 --> 00:29:58.079
So they urged companies to do these tests, right?
00:29:58.160 --> 00:30:16.400
But as we know from the history of Facebook, sometimes they don't want to look too closely into things depending on what they find, or you know, testing things on vulnerable users brings a lot of scrutiny, a lot of negative scrutiny, and requires like IRB approval, all of these things.
00:30:16.559 --> 00:30:21.920
So we just haven't seen this level of testing on this particular issue.
00:30:22.079 --> 00:30:52.000
And I would say also that like part of part of the reason that even though it seems so obvious when you explain it this way, like a consumer product going the way of optimization, especially an ad-supported one, I think part of the reason that the industry as a whole didn't prioritize this issue is because of the way that the AI safety community, like quote unquote AI safety community, is much
more oriented around, you know, what they perceive as risks of super advanced AI, right?
00:30:52.079 --> 00:30:57.680
So they did, they did test on things like sycophanecy and highlight some of these problems.
00:30:57.839 --> 00:30:59.440
They call it reward hacking, right?
00:30:59.519 --> 00:31:05.039
Like you reward the model for getting the thumbs up or thumbs down from the user.
00:31:05.279 --> 00:31:12.880
And they did notice that these things went awry, but people weren't testing it for necessarily this kind of use case, right?
00:31:13.039 --> 00:31:16.400
Or they weren't testing it in, you know, for regular conversations.
00:31:16.559 --> 00:31:27.599
They were looking at like hyper persuasion from a super advanced AI or, you know, ways that the model could deceive you, which I think kind of anthropomorphizes it in a way, right?
00:31:27.920 --> 00:31:40.480
And they're not necessarily looking at multi-turn, you know, so they're looking at like one back and forth and not like you using it an hour a day or two hours a day, five days a week, that sort of thing.
00:31:40.640 --> 00:31:48.559
So I think like this very pedestrian way that things could go awry just hadn't been looked at that closely.
00:31:49.039 --> 00:32:21.440
So basically what you're saying, you know, to a certain degree is that because these researchers and because there is this community within these AI companies and this broader kind of AI research community that believe that the large language models and chatbots are like one step toward an artificial general intelligence where you know the computers are going to gain sentience and the ability to think for themselves that this is leading them to not really prioritize these potential like real-world impacts because their focus is just somewhere else completely.
00:32:21.519 --> 00:32:36.480
Like I guess it makes that that conversation more difficult to have because I feel like you know, when I hear the discussion of these potential outputs coming from chatbots, the question becomes are we seeing emergent properties, right?
00:32:36.799 --> 00:32:42.000
Or are we seeing these things starting to think for themselves and mislead people?
00:32:42.240 --> 00:32:57.039
And that like inherently shifts the conversation away from are these products, you know, having these outputs that are harmful for people and more like, oh my God, are we starting to see the glimmerings, the beginnings of sentience that is just emerging here?
00:32:57.200 --> 00:32:58.720
Like, is that part of the issue there?
00:32:59.039 --> 00:33:06.240
You know, I'm trying to glean like why they prioritize what kinds of research from the outside.
00:33:06.400 --> 00:33:08.640
So I can't say exactly, right?
00:33:08.720 --> 00:33:14.079
Like maybe researchers really want to test this, but they get more pressure to not make those papers public.
00:33:14.160 --> 00:33:18.880
But I'll just say, like, I have been interested in this issue and looking at it for
a while.
00:33:19.200 --> 00:33:27.680
And when I look at like Anthropic has great papers on reward hacking and sycophancy, but they did not, they don't delve into this particular use case.
00:33:27.759 --> 00:33:30.319
And I, you know, it just makes me wonder why.
00:33:30.480 --> 00:33:40.799
And I know that because I'm interested in this and have written about some of these issues, sometimes researchers come to me and I know that they care about these issues and want to see more research around it.
00:33:40.960 --> 00:33:48.160
So when you look at the reward hacking and sycopency papers, you're not seeing, you know, focus on these particular issues.
00:33:48.400 --> 00:33:57.119
There could be a variety of reasons why I think it's undeniable that the kind of focus of AI safety has shifted it in one direction.
00:33:57.279 --> 00:34:06.720
If you look at like open AIs, like when they look at potential long-term risks, it's it's stuff that sounds a lot more sci-fi, right?
00:34:06.799 --> 00:34:18.800
Or like is the subject of existing concern for effective altruists and the AI safety community, the rationalist community looking at risk from bioweapons, risk from super persuasion.
00:34:18.880 --> 00:34:22.159
So, like, you know, somebody using it to shift an election or something.
00:34:22.320 --> 00:34:24.159
They're not looking at these particular risks.
00:34:24.320 --> 00:34:38.960
And to me, it just seems so obvious that subtle manipulation of our minds and our interior life and you know, the vulnerability that comes when you when you promote the use of the chatbot for therapy.
00:34:39.199 --> 00:34:41.440
The like, why, why isn't this issue prioritized?
00:34:41.519 --> 00:34:47.039
But again, I'm just, you know, this is me trying to understand why I'm seeing the papers that I'm seeing.
00:34:47.360 --> 00:34:48.000
No, definitely.
00:34:48.320 --> 00:34:49.119
And I appreciate that.
00:34:49.199 --> 00:34:56.559
I I feel like me personally, I I've often found it kind of frustrating when we start to talk about these real issues, you know, with the chatbots.
00:34:56.639 --> 00:35:23.280
And before you can even get to talking to them, you need to kind of like diffuse and unpack these notions of the chatbot being sentient and trying to explain that that is not what's actually happening here, just to start to have like a baseline conversation about what's happening, which uh I feel like really kind of sets back any discussion of these real harms that we're talking about that can that can come of the way that these chatbots have been created and and manipulated and the way that the weights have been set and things like that.
00:35:23.599 --> 00:35:23.760
Yeah.
00:35:23.840 --> 00:35:28.480
And you know, some of it is just like jargon, which all researchers are prone to, right?
00:35:28.559 --> 00:35:36.719
But like reward hacking from user feedback and sycophancy, it just doesn't, you know, it doesn't bring that immediacy to mind.
00:35:36.880 --> 00:35:51.280
I will say, like Mike Krieger, who is the ex-co-founder of Instagram, who now works at Anthropic, he was recently on Hard Fork and he mentioned that there was an internal essay about why you shouldn't optimize for engagement.
00:35:51.360 --> 00:36:00.159
And we haven't, in recent times, we haven't seen people showing these kinds of examples that they showed from OpenAI or from character AI from Anthropic.
00:36:00.239 --> 00:36:02.400
But you know, they didn't publish that essay.
00:36:02.559 --> 00:36:17.039
I would love to see it, but you know, I do think probably some of it has to do with companies also clamping down on publicizing research, you know, in a lot of cases because it's competitive, but I have to imagine in some cases also because it's controversial.
00:36:17.360 --> 00:36:18.239
Yeah, no, that makes sense.
00:36:18.320 --> 00:36:22.559
And hey, if there's someone at Anthropic who has access to this, they can go ahead and leak it to Natasha.
00:36:22.960 --> 00:36:23.840
Please send it to me.
00:36:24.000 --> 00:36:26.239
My signal is Natasha.10.
00:36:26.960 --> 00:36:28.000
Now you know what to do.
00:36:28.079 --> 00:36:28.320
Yeah.
00:36:28.480 --> 00:36:30.320
Hopefully it arrives in your inbox soon.
00:36:30.559 --> 00:36:47.920
I I feel like, you know, when we talk about these things, you know, when I hear what you're describing, I think about this study that you cited in your piece from OpenAI that was talking about how people using these chatbots tend to be lonelier, have a greater emotional dependence on them, have lower socialization with other people.
00:36:48.159 --> 00:36:58.000
And that also brings to mind these comments that Mark Zuckerberg was making recently about, you know, wanting to give people a load of AI friends because they don't have enough friends in their real life.
00:36:58.239 --> 00:37:13.440
And there's obviously something that the companies are doing here that they're responding to where it's like, yeah, we do live in a society where there is this degree of loneliness where it feels like certain people do not have, you know, the social connections that they would want to have or or that would feel adequate.
00:37:13.760 --> 00:37:36.400
But on the flip side of that, it's like, are these tools really making that any better when they are, you know, trying to create these really intimate experiences that feel like, you know, they're almost tricking people as to the, you know, the type of thing that they're actually interacting with in some cases and whether it's really properly understanding with them and engaging with them and being a friend or a therapist or what have you.
00:37:36.559 --> 00:37:40.480
Yeah, I don't know if you have any thoughts on the way that the companies are are approaching that.
00:37:40.719 --> 00:37:50.079
Oh, yes, I have uh I mean, I would just advise everybody to listen to Mark Zuckerberg's interview on the Dwarkeshes podcast.
00:37:50.320 -->
00:37:57.519
It is wild to me, the portion where he's talking about using using these chatbots as friends.
00:37:57.679 --> 00:38:16.320
I mean, he talks about, well, first of all, let me let me back up and just say that like in talking to users, in paying a lot of attention to user forums, I see that people do find this extremely valuable, especially for use in mental health and loneliness and you know, in some cases for trauma.
00:38:16.480 --> 00:38:26.719
Um, that company that I talked about, Chai, they used to have like on their pitch deck or on their promotional materials, like a testimony from a user who said they used it for eating disorders.
00:38:26.800 --> 00:38:32.719
And Chai has put like a bunch of its conversations online, and you can see there's a lot of eating disorder stuff in it.
00:38:32.960 --> 00:38:34.559
There's a lot of other stuff as well.
00:38:34.880 --> 00:38:37.920
But you have people who are finding real utility in it.
00:38:38.000 --> 00:38:42.639
But I think if you're promoting that use case, you should also be testing that use case, right?
00:38:42.719 --> 00:38:44.880
And we're not seeing that, at least publicly.
00:38:45.039 --> 00:38:56.159
And so you have Mark Zuckerberg talking about how, like, he understands that people might find it off-putting, but you have to look at the quote unquote revealed preferences of users.
00:38:56.400 --> 00:39:03.440
And it feels like code for, you know, he talks about the North Star of the data that that Meta is looking at.
00:39:03.599 --> 00:39:06.960
And it sounds like he's talking about optimizing for engagement.
00:39:07.039 --> 00:39:09.760
I mean, that's how other people read this interview as well.
00:39:09.920 --> 00:39:11.599
And he's just being really bold about it.
00:39:11.760 --> 00:39:23.920
And if you look, there's there's interviews from like back in 2019 or 2020 where he's talking about the challenges of this because Meta or Facebook then had been called out for using revealed preferences.
00:39:24.079 --> 00:39:38.639
Like if you judge time spent on a conversation, you also have to judge like the entire power of Meta and every knob and dial that it can tweak to manipulate you to spend more time, you know, talking to a chatbot.
00:39:38.880 --> 00:39:41.199
I mean, he was just really bold about it, right?
00:39:41.360 --> 00:39:46.800
I mean, he made this comment about like most people have three friends, but demand for 15 more.
00:39:46.880 --> 00:39:49.280
And he sounds like robotic, right?
00:39:49.360 --> 00:39:57.199
And he he kind of drew out this really dystopic scenario about like where you're basically just talking to chatbots all day.
00:39:57.440 --> 00:40:05.440
But to couple that with like our North Star is revealed preferences, I think it just shows you that it sounds a lot like what happened with the social media era.
00:40:05.679 --> 00:40:06.880
I definitely agree with that.
00:40:07.039 --> 00:40:12.159
And there were a couple interviews he
gave around that time that felt really like defiant, right?
00:40:12.239 --> 00:40:27.360
You know, that that we are going to do this thing that we think is good for the world, regardless of whether it is or not, because we can see all the other times when Meta was supposedly, or Facebook before it was supposedly doing things that were great and then ignoring the potential consequences that it just didn't want to see.
00:40:27.519 --> 00:40:36.320
And it feels like once again, you know, Zuckerberg is doing a very similar thing with AI as he tries to capture this market and make sure that it has a good market share there.
00:40:36.480 --> 00:40:50.079
When you talk about these people who are using these products and these chatbots, you talked earlier about how some people are engaging with chatbots that are like anime characters and, you know, other characters in pop culture that they might like.
00:40:50.239 --> 00:40:59.840
But then on the other hand, there are people engaging with these things where the AI is kind of positioned as being a companion or a friend or even a therapist.
00:41:00.079 --> 00:41:14.559
After you know, looking at the way that people talk about these chatbots, do you have an idea the degree to which people, you know, know that the technology they are engaging with isn't, you know, intelligent in the way that it might appear?
00:41:14.719 --> 00:41:22.239
Or to a certain degree, do you feel like people are being misled about what they're actually engaging with and where their conversations are going?
00:41:22.639 --> 00:41:30.800
Well, first of all, I would say the therapy, companionship, friend use case is often all in the anime chatbot.
00:41:30.960 --> 00:41:49.039
Like, you know, young people or people who like anime or like pop culture characters find it like a universal kind of feedback that you hear, at least from companies, they say that people feel a lot more comfortable talking to something that they perceive as a friend or a familiar character.
00:41:49.199 --> 00:41:55.920
That's maybe those characters are getting more usage than the, you know, because they also have like AI therapist characters.
00:41:56.159 --> 00:41:57.840
So often it's one and the same.
00:41:58.000 --> 00:42:10.480
And we saw with another character AI lawsuit where this minor committed suicide, died by suicide, sorry, he was talking to the same chatbot, this like Daenerys Targaryen figure from Game of Thrones.
00:42:10.639 --> 00:42:20.159
And sometimes it was therapy, and sometimes it was romance, and sometimes it was familial, but it was over time developing this closeness and this trust in the chatbot.
00:42:20.239 --> 00:42:22.960
And I I've seen some people say, like, well, it's just text.
00:42:23.119 --> 00:42:24.719
Like, what are you so worried about?
00:42:24.960 --> 00:42:33.599
But I think that you can see when you look at the screenshots, you know, you're talking to somebody that's not fully developed, right?
00:42:33.679 -->
00:42:42.400
Like you're talking to them at the time that they are starting to learn how to have relationships and talk about these issues and talk about emotions.
00:42:42.639 --> 00:42:54.000
And, you know, so I would just say to the CEOs that are invoking the loneliness epidemic as a reason that their products should be marketed or pushed in this way.
00:42:54.159 --> 00:42:56.639
I would just again say, like, are you testing for this?
00:42:56.719 --> 00:43:00.079
Are you seeing how they respond to deeply lonely users?
00:43:00.400 --> 00:43:10.320
And it's really fascinating because it, you know, there genuinely is a loneliness epidemic and there is like a lack of care and access to care.
00:43:10.719 --> 00:43:20.559
But you are having trillion-dollar companies go into this market in a way that at least we can all say there's no oversight, right?
00:43:20.719 --> 00:43:28.559
Like these character AI lawsuits are the only reason that we're seeing these examples of vulnerable people being manipulated.
00:43:28.960 --> 00:43:32.880
Yeah, I I just think it's so, you know, concerning to hear that.
00:43:33.039 --> 00:43:36.079
And on that point, like we just talked about Mark Zuckerberg.
00:43:36.320 --> 00:43:45.199
Do we see actions from whether it's the major companies like Meta, Google, OpenAI, or the smaller ones that you're talking about, you know, character AI, Chai?
00:43:45.519 --> 00:43:48.800
Obviously, replica was one that was getting a lot of attention for a while.
00:43:49.119 --> 00:43:53.840
Do we see actions by these companies to actually try to address some of these issues?
00:43:54.000 --> 00:43:57.679
Or, you know, are they really trying to ignore that this is happening?
00:43:58.079 --> 00:44:10.719
I think you can point to the OpenAI issue Study with MIT, the, you know, where they used actual human people, they tested them for four weeks as a sign that at least OpenAI is putting this research out there.
00:44:10.800 --> 00:44:15.840
It's the first of its kind that we've seen on this topic with humans, you know, recently.
00:44:16.079 --> 00:44:27.599
I know that Google, like part of the reason I brought up that Blake Lemoyne thing in terms of being able to play around a little bit with Lambda, is that Google just opened up its Gemini app to children.
00:44:27.840 --> 00:44:30.559
So you see that it's been almost three years.
00:44:30.719 --> 00:44:34.639
They were definitely experimenting with it in some way back then.
00:44:34.800 --> 00:44:37.440
And only now are they releasing it to children.
00:44:37.599 --> 00:44:45.920
So I think that that shows that there's probably been, you know, some internal concern, right, about releasing it in a certain way.
00:44:46.079 --> 00:44:53.440
But I think also when you see adoption, like widespread adoption, like if you look at, you know, what are the most popular consumer apps?
00:44:53.519 --> 00:44:58.639
It's often companions or AI homework help or, you know, chatbots in some way.
00:44:58.880 --> 00:45:08.079
So you have like companies that I'm sure would rather not be scrutinized and rather not be harming children at the same time facing a lot of commercial pressures.
00:45:08.400 --> 00:45:17.280
So it's only whistleblowers, it's only lawsuits, it's reports from consumers that are holding companies accountable.
00:45:17.440 --> 00:45:21.679
There's there's nobody else looking at what they're doing behind the scenes to optimize.
00:45:22.000 --> 00:45:24.480
Yeah, which is so often, I guess, what we see, right?
00:45:24.559 --> 00:45:31.840
Before there has to be this broader conversation, there are people who are looking into it to eventually try to force the companies to do something about it.
00:45:32.000 --> 00:45:35.519
You mentioned earlier in our conversation about business models, right?
00:45:35.599 --> 00:45:45.840
We know that these companies and their AI divisions are often losing a lot of money right now, you know, because of the costs of actually running these models, training these models, all these sorts of things.
00:45:46.000 --> 00:45:49.360
Obviously, they are trying to get closer to profitability.
00:45:49.519 --> 00:45:53.199
What are we seeing in the business models applied to these chatbots?
00:45:53.360 --> 00:45:59.360
You know, the degree to which they're trying to implement advertising and other forms of monetization.
00:45:59.599 --> 00:46:07.519
And what impact do you think a greater push for profitability will have on, you know, the way that these tools are ultimately governed?
00:46:07.840 --> 00:46:10.719
Well, I think we haven't seen anything yet.
00:46:10.880 --> 00:46:14.480
It's not, it's it's barely even been the first step.
00:46:14.639 --> 00:46:24.880
Like I saw that OpenAI executive recently said like he and Sam decided on$20 a month because it sounded like what they thought people would pay.
00:46:25.199 --> 00:46:33.760
So even with the pricing issue for subscriptions, because obviously that's another way that they can make money and are making money right now, and people are paying.
00:46:33.920 --> 00:46:40.320
I mean, it, you know, they're losing money, but they're also kind of racking it up in the billions in terms of revenue.
00:46:40.480 --> 00:46:42.559
It's just ability to do that long term.
00:46:42.719 --> 00:46:44.079
But we haven't seen anything yet.
00:46:44.159 --> 00:46:47.119
There's not been an ad in one of these chatbots.
00:46:47.280 --> 00:46:52.239
But you you do see them hiring these people, right, who have specialized in this.
00:46:52.559 --> 00:46:59.920
So I mean, just rewind back 20 years of Google and look at where Google is now.
00:47:00.079 --> 00:47:02.639
I think we are gonna see the exact same things, right?
00:47:02.719 --> 00:47:07.760
Like people are using these chatbots for search, they're using it as their portal to the web.
00:47:08.079 --> 00:47:17.119
So everything that we've seen happen to Google and to Facebook, I imagine we're
going to see happen in chatbots.
00:47:17.280 --> 00:47:24.960
Like a lot of these issues that I'm talking about, loneliness, mental health epidemic, like it's not happening anew with chatbots.
00:47:25.039 --> 00:47:28.960
It's happening when you have people who are already primed for this, right?
00:47:29.039 --> 00:47:31.840
Like they're already on their smartphone.
00:47:31.920 --> 00:47:39.360
And, you know, there's not even been enough studies to know like whether or not it is addiction or how, you know, sometimes it does help people.
00:47:39.440 --> 00:47:41.280
It does make people feel less lonely.
00:47:41.519 --> 00:47:50.960
So into that morass, you are having like a bunch of little closed wall versions of these major companies.
00:47:51.119 --> 00:47:54.719
And in some cases, like in Google and Meta, it's the same companies, right?
00:47:54.960 --> 00:47:56.800
So I don't think we've seen anything yet.
00:47:57.119 --> 00:48:14.719
Yeah, you know, the move fast and break things is not often the best place to be uh thinking about the safety and making sure that these tools are not going to have the harmful impacts that we're already seeing out there, and that I'm sure there are many things that we're not even seeing because not a ton of research is being done on it or not nearly enough, right?
00:48:15.039 --> 00:48:23.119
Well, yeah, and I think we're seeing companies like I think it's a real test now of how much responsibility they want to take for the outputs.
00:48:23.280 --> 00:48:30.559
You know, you can see like Google and Character AI just move to have the lawsuit where I mentioned the kid cutting himself.
00:48:30.719 --> 00:48:32.800
They push to move it to arbitration.
00:48:32.880 --> 00:48:36.400
So private arbitration, that means much less public scrutiny, right?
00:48:36.719 --> 00:48:46.079
And you see through the New York Times lawsuit recently of OpenAI for copyright infringement, they're asking them to keep their user chats.
00:48:46.320 --> 00:48:52.000
And that has helped bring more attention to the amount of data that they have kept on users.
00:48:52.320 --> 00:49:03.679
So again, all of these issues around privacy and surveillance and how your own data is being used against you are, I feel like we're just at the start of it.
00:49:03.920 --> 00:49:21.440
Yeah, the data question is such a big one that I feel like doesn't even get, you know, nearly enough consideration on the one hand, you know, have so much user data was already used to train these chatbots, but now how much they're they're keeping on people and you know going to be using to tailor, you know, the responses and things like that down the road.
00:49:21.679 --> 00:49:26.960
But I I did have one final question for you, kind of picking up on some of the things that you were just saying there.
00:49:27.199 --> 00:50:01.840
You know, when you were saying that Google had opened its Gemini to children, I was struck by thinking about how in many countries
right now, we're in the process of having this debate about actually raising the minimum age that kids and teenagers should be able to access social media at all, as there is this growing discussion about, you know, the consequences of social media that maybe we haven't been wanting to talk about for the past number of years, or, you know, we've even seen get worse because of these pressures toward commercialization and increasing the profits that companies can extract from these platforms.
00:50:02.079 --> 00:50:25.119
I guess, you know, based on some of the things that I've been hearing you saying through the conversation, do you think that we're headed to a point where chatbots are going to start presenting this whole new range of concerns that are going to be similar to social media and that we're going to have to tackle anew at a time when we're just seemingly trying to kind of figure out what proper governance of social media should look like in the first place?
00:50:25.360 --> 00:50:26.320
Yes, I think so.
00:50:26.480 --> 00:50:31.760
I mean, this has been the a theme throughout my work that I've been trying to highlight.
00:50:31.920 --> 00:50:35.599
You know, there hasn't been that much research on it and public data on it.
00:50:35.679 --> 00:51:02.559
But yes, I very much think we're going to be going through a lot of the same issues that we went through with social media, where it is a product manager in Mountain View, in Menlo Park, making a seemingly benign decision or, you know, a decision for commercial reasons that then will have impacts that we might not see for years to come in different markets, in different types of users.
00:51:02.719 --> 00:51:14.800
You know, one thing about Google opening up Gemini to kids is the reason we found out about it is because they emailed parents who have like a Google link program where you have like some kind of parental controls.
00:51:15.119 --> 00:51:20.880
And on the one hand, you know, they're saying, like, hey, just FYI, next week we're opening it up to kids.
00:51:21.039 --> 00:51:28.159
And I imagine that they want to do it for a AI homework usage because that's such a popular use case.
00:51:28.320 --> 00:51:29.599
Kids are already doing it.
00:51:29.840 --> 00:51:38.239
Would you rather have them do it with like an app that's owned by Chinese developers or Google that has more privacy restrictions in place, right?
00:51:38.400 --> 00:51:45.280
But if you read the email, they're saying like, please talk to your kids about how this is not really a human.
00:51:45.679 --> 00:51:49.119
Please talk to your kids about how you can't trust the answers for that.
00:51:49.280 --> 00:52:02.960
And it just to me highlights like, you know, we don't even have enough digital literacy to be able to contend with the ways that social networks, you know, work on our human intuition and human instincts.
00:52:03.039 --> 00:52:10.800
And now we have chatbots like where is a parent supposed to go to figure out how to explain this to their child?
00:52:11.119 --> 00:52:22.880
You know, I'm not seeing these same companies put out really helpful information into how the levers of power that they're pulling behind the scenes, the ways they train it, the ways it can go wrong outside.
00:52:23.119 --> 00:52:34.000
And at the same time, I'm not seeing media do a great job of that either, like explaining always how these things work, you know, partly because there is a lack of transparency.
00:52:34.239 --> 00:52:40.239
But to me, yeah, it's just, it feels like we're speedrunning the last couple decades of tech.
00:52:40.559 --> 00:52:41.440
Yeah, it's so grim.
00:52:41.519 --> 00:52:59.360
And even as you describe the contents of that email, like the contrast between how they're framing this rollout to children compared to how so many of the executives actually talk about these technologies in public and the huge difference that is in those two different framings of what generative AI is actually doing and what the chatbots actually are.
00:52:59.599 --> 00:53:01.840
Natasha, keep up the great reporting.
00:53:01.920 --> 00:53:04.559
I really enjoyed this piece and having you back on the show.
00:53:04.719 --> 00:53:11.280
I think that this is going to be such an important topic, and I'm sure we'll have you back on the show to discuss it further as you continue, you know, kind of reporting on this.
00:53:11.440 --> 00:53:13.760
Thanks so much for taking the time to speak with me today.
00:53:14.000 --> 00:53:15.039
Thanks for asking me.
00:53:15.119 --> 00:53:17.199
I I always love coming on the show.
00:53:19.119 --> 00:53:22.159
Natasha Tiku is a technology reporter at the Washington Post.
00:53:22.320 --> 00:53:26.239
TechWon't Save Us is made in partnership with The Nation magazine and is hosted by me, Paris Marks.
00:53:26.400 --> 00:53:27.599
Production is by Kylie Hewson.
00:53:27.679 --> 00:53:32.320
TechWon't Save Us relies on the support of listeners like you to keep providing critical perspectives on the tech industry.
00:53:32.480 --> 00:53:37.280
You can join hundreds of other supporters by going to patreon.com slash techwon'save us and making a pledge of your own.
00:53:37.440 --> 00:53:38.079
Thanks for listening.
00:53:38.239 --> 00:53:39.920
Make sure to come back next week.