WEBVTT
00:00:04.080 -->
00:00:06.320It's not something we're used to doing with our computer.
00:00:06.320 -->
00:00:08.480So I think this is kind of new.
00:00:08.960 -->
00:00:15.118The whole rhetoric around the power of AI is about how it's the way that we work.
00:00:15.519 -->
00:00:21.518Once you type your data in and you press that ender key, you're setting your data free, and you know, you've kind of lost control of that.
00:00:23.760 -->
00:00:24.239Hello world.
00:00:24.239 -->
00:00:26.239Welcome to the Tech Overflow podcast.
00:00:26.239 -->
00:00:27.678I'm Hannah Clayton Langton.
00:00:28.239 -->
00:00:33.118And I'm Hugh Williams, and we are the podcast that explains technology to curious people.
00:00:33.118 -->
00:00:34.000How are you, Hannah?
00:00:34.000 -->
00:00:34.719How's life?
00:00:35.200 -->
00:00:35.759Life's great.
00:00:35.759 -->
00:00:39.039I'm excited to revisit our favorite topic of AI today.
00:00:39.039 -->
00:00:39.920Um, how are you?
00:00:40.240 -->
00:00:40.640Oh, I'm well.
00:00:40.640 -->
00:00:41.118I'm well.
00:00:41.118 -->
00:00:48.240I've been in lots of prep for this episode, including my own use of ChatGPT and Claude and uh all kinds of things, reading some research papers.
00:00:48.240 -->
00:00:51.840It's been uh it's been a ton of fun, but um really pumped to get into this episode.
00:00:51.840 -->
00:00:53.679And as you say, great to be back on AI.
00:00:53.679 -->
00:00:57.679I mean, we did those two fabulous episodes in season one.
00:00:57.679 -->
00:01:08.879If you want to check those out, they're episodes six and And then one of our best rating episodes was episode two in this season, where we talked about vibe coding and agentic So it's just great to be back on the theme, Hannah.
00:01:09.438 -->
00:01:10.159Yeah, love the theme.
00:01:10.159 -->
00:01:16.959I find that the AI topics are the ones where I feel like my chemistry changes the most because I learned so much.
00:01:16.959 -->
00:01:22.239Today I'm very curious for us to get into how people are using AI.
00:01:22.239 -->
00:01:27.519And I guess when I say AI in this context, I really mean LLMs, like I think a lot of people do.
00:01:27.519 -->
00:01:29.519So, how are you using it?
00:01:29.519 -->
00:01:30.959How should I be using it?
00:01:30.959 -->
00:01:35.760Uh, I think we've got some external sort of stats on how other people are using it.
00:01:35.760 -->
00:01:39.760And should we be worried about the ways that we are using it?
00:01:40.078 -->
00:01:47.438And as your resident expert, Hannah, I'm I'm really keen unpack what happens when you type data into one of these Where does your data go?
00:01:47.438 -->
00:01:48.400How is your data used?
00:01:48.400 -->
00:01:49.438Where is your data stored?
00:01:49.438 -->
00:01:50.959How long is it stored for?
00:01:50.959 -->
00:01:55.920And maybe give our listeners a little bit of advice and around sort of how to how to think about that data.
00:01:55.920 -->
00:01:57.840So I think this will be a fun, this will be a fun episode.
00:01:58.400 -->
00:01:58.400100%.
00:01:58.400 -->
00:02:03.200And our first episode of this season gave me pause for around my data in general.
00:02:05.200 -->
00:02:07.040friends that listened that they felt the same way.
00:02:07.040 -->
00:02:10.878So uh we're about to make that existential crisis a whole lot worse, guys.
00:02:10.878 -->
00:02:11.680Talk about LLMs.
00:02:11.680 -->
00:02:18.400Well, when it comes to how I'm using AI, I did prepare earlier, which maybe be a good place to kick us off.
00:02:18.400 -->
00:02:21.199So I actually asked ChatGPT how I use ChatGPT.
00:02:21.199 -->
00:02:23.520I asked for it to summarize it.
00:02:23.520 -->
00:02:26.318Um, I don't know if that's allowed, but here we are.
00:02:26.318 -->
00:02:28.800And it gave me my top three use cases.
00:02:28.800 -->
00:02:31.199So I will walk you through them if you're interested.
00:02:32.158 -->
00:02:32.400Very interesting.
00:02:33.038 -->
00:02:35.199Okay, so first one, everyday decision support.
00:02:35.199 -->
00:02:37.199That's quite vague and not a huge surprise.
00:02:37.199 -->
00:02:40.960Makes practical decisions faster, outsourcing research.
00:02:40.960 -->
00:02:42.479Actually, I just bought some new running shoes.
00:02:42.479 -->
00:02:44.479So ChatGPT helped me choose them.
00:02:44.479 -->
00:02:50.560Second related theme is health and performance curiosity, exercise physiology, bit of cold water swimming.
00:02:50.560 -->
00:02:55.438If you've listened to earlier episodes, you know that I I was curious about cold water swimming.
00:02:55.919 -->
00:02:56.080In London?
00:02:56.080 -->
00:02:56.639Seriously?
00:02:57.680 -->
00:03:05.120Hugh, in London in March, I went for at least 60 seconds in water.
00:03:05.120 -->
00:03:06.158It was really cold.
00:03:06.639 -->
00:03:07.520That's that's ice bar territory.
00:03:07.520 -->
00:03:08.800Anyway, okay, go henna.
00:03:08.800 -->
00:03:09.360Yeah.
00:03:09.919 -->
00:03:12.960The third one, which is probably the most flattering, is exploration.
00:03:12.960 -->
00:03:21.120So it says here economic or career topics, language and and AI concepts, no surprise there.
00:03:21.120 -->
00:03:29.919So top three, I think that's a fairly accurate of myself from my trusty in-pocket decision helper, aka
00:03:30.800 -->
00:03:31.280That's a great set.
00:03:31.280 -->
00:03:33.840You should be very, very proud of that third bucket, language and etymology.
00:03:33.840 -->
00:03:35.038Look at you go.
00:03:35.038 -->
00:03:35.758That's awesome.
00:03:36.479 -->
00:03:41.598I do study languages at university, so there were I didn't that I was actually still thinking in that way, but I must be.
00:03:41.598 -->
00:03:41.840Yeah.
00:03:42.479 -->
00:03:52.878I didn't do the same homework, but I'll tell you what I do, Hannah, and maybe our listeners can follow along with If you've got ChatGPT installed on any device, you can go into the settings.
00:03:52.878 -->
00:04:01.919And if you scroll down, you'll see an option called If you click on that and then you scroll down again, you are going to see an option called memory.
00:04:01.919 -->
00:04:12.080If you click on that, and then you click again on manage you'll see a bunch of memories that Chat GPT has from all of the sessions that you've had with ChatGPT.
00:04:12.080 -->
00:04:14.878So these are things that it thinks about you.
00:04:14.878 -->
00:04:22.319And um, I scanned these and I came up with my top three Bit of overlap, I think, Hannah.
00:04:22.800 -->
00:04:29.519Okay, and just to be clear, these aren't prompts that it's These are like snippets of information it's gleaned about you as a person.
00:04:30.000 -->
00:04:30.319Yeah, correct.
00:04:30.319 -->
00:04:32.000So these are sort of meta-level summaries.
00:04:32.000 -->
00:04:35.680So it's a little bit like the prompt that you ran, Hannah, to find out the top three categories.
00:04:35.680 -->
00:04:40.000If you like, these are sort of meta reflections on the of things that I ask.
00:04:40.000 -->
00:04:46.240Maybe later on in the show we can talk about how these are behind the scenes in ChatGPT, but uh, you know, maybe not important for now.
00:04:46.240 -->
00:04:54.720But uh it looks like if I if I bump up a level and try and the top three, I would say I spend a lot of time talking to it about music and musicians.
00:04:54.720 -->
00:04:55.759Very cool.
00:04:55.759 -->
00:04:59.680And a lot of time talking about recipes, nutrition.
00:04:59.680 -->
00:05:08.160So for example, it's figured out that I don't like eggs I like a breakfast smoothie with increased yogurt and Sounds true.
00:05:08.160 -->
00:05:12.560So lots of optimization, things about cafe lattes, all kinds of very Melbourne things.
00:05:12.560 -->
00:05:19.600And then a good amount of uh, you know, fitness running stuff, including lots of memories about park run.
00:05:19.600 -->
00:05:26.560Because I love a park run on a Saturday and how many park runs I've done and where I've done them, and suggestions for park runs and all those kinds of things.
00:05:27.040 -->
00:05:28.399That sounds pretty accurate, to be honest.
00:05:28.399 -->
00:05:32.000So you're using it for everyday queries, it sounds like.
00:05:32.319 -->
00:05:33.040Yeah, yeah, yeah.
00:05:33.040 -->
00:05:36.160And it's uh it's got quite a few memories in here that I probably delete.
00:05:36.160 -->
00:05:39.120So it's um it says that I'm going to London for three months at the end of June.
00:05:39.120 -->
00:05:40.160I think that was two years ago.
00:05:40.160 -->
00:05:41.600So I could probably safely delete that.
00:05:41.600 -->
00:05:45.040But you know, there's a there's a few things in here that uh off base, as it were.
00:05:45.439 -->
00:05:52.319Okay, so you can delete the memories and they sort of form there and sit there and reinforce the output.
00:05:52.319 -->
00:05:57.680So like when we were coming to Melbourne, I asked for travel and it was like, oh, do you want me to recommend some hikes?
00:05:57.680 -->
00:06:00.160And that's probably because it's worked out that I like hiking.
00:06:00.879 -->
00:06:01.199Yeah, absolutely.
00:06:01.199 -->
00:06:15.278And and likely what's happening behind the scene here is when you type in a prompt into ChatGPT, behind the scenes, secretly appending these memories, and then these memories used in the in the context window in the LLM to produce answer.
00:06:15.278 -->
00:06:21.120So it makes it feel a little bit more like it knows you, I when it's when it's giving you back the answers using memories.
00:06:21.759 -->
00:06:23.920It's pretty cool that you can go in and see them.
00:06:23.920 -->
00:06:25.278So I will go and do that.
00:06:25.278 -->
00:06:30.319I'm not gonna do that live in case it's something I don't wish to reveal, but I'll go and do it after the episode.
00:06:30.800 -->
00:06:31.600Yeah, be careful with it.
00:06:31.600 -->
00:06:34.879Uh there's definitely some personal stuff in there that I gonna read out on here, Hannah.
00:06:35.278 -->
00:06:36.399Yeah, Hannah, you are a hypochondriac.
00:06:36.399 -->
00:06:37.680Okay.
00:06:37.680 -->
00:06:44.800Um, Q, from the research that I did ahead of the show, it sound like either of us is dissimilar to the norm.
00:06:45.040 -->
00:07:00.000So most people are using Chat GPT for the same sort of information queries and not necessarily using it for automation of tasks or coding, which is the U case, the use case you hear a lot about.
00:07:00.000 -->
00:07:03.120But I'm like asking it what running shoes to buy, and so are people.
00:07:03.519 -->
00:07:04.079Yeah, that's it.
00:07:04.079 -->
00:07:08.959Um, there was a paper that came out of OpenAI, I think September last year, sort of a white paper.
00:07:08.959 -->
00:07:12.160So not a peer-reviewed piece of research, but a white paper that came out.
00:07:12.160 -->
00:07:20.720And uh first thing is that almost 80% of conversations into three buckets practical guidance, seeking and writing.
00:07:20.720 -->
00:07:32.560And so I think you and I, lots of practical guidance, lots of seeking information, but we probably didn't have in our top three writing, you know, so whether it's refining things, emails, you know, improving documents.
00:07:32.560 -->
00:07:37.920And they also interestingly proposed a new about how these tools are used.
00:07:37.920 -->
00:07:41.838And it's the asking, doing, expressing framework that up with.
00:07:41.838 -->
00:07:47.040And what they figured out was that 49% were asking a question and getting back an answer.
00:07:47.040 -->
00:07:57.199Uh 40% of the prompts were getting it to do something, uh, then 11%, and this is super interesting, I know you love Uh, 11% is people just expressing something.
00:07:57.199 -->
00:08:01.838So that's neither asking a question nor asking it to do it's just sharing a feeling with ChatGPT.
00:08:02.959 -->
00:08:06.160Okay, so to put that into real terms, let's take a cooking use case.
00:08:06.160 -->
00:08:08.560Asking is like a simple question.
00:08:08.560 -->
00:08:12.000So it might be like, is there cheese in pesto?
00:08:12.959 -->
00:08:12.959Yep.
00:08:13.360 -->
00:08:13.360Right.
00:08:13.600 -->
00:08:13.600100%.
00:08:14.399 -->
00:08:19.519And then doing would be like, create me a dairy-free pesto
00:08:20.240 -->
00:08:20.240Yeah.
00:08:20.959 -->
00:08:22.879And then expressing would be.
00:08:23.600 -->
00:08:24.079I love the pesto.
00:08:24.720 -->
00:08:25.439Yeah, and that's it.
00:08:25.439 -->
00:08:27.600I mean, I don't I think the expressing is the thing I'm doing the least.
00:08:27.600 -->
00:08:29.120It's kind of hard for me to get my head around.
00:08:29.120 -->
00:08:32.960But you said 11% of interactions are that expressing
00:08:33.278 -->
00:08:33.519Yeah, exactly.
00:08:33.519 -->
00:08:39.600People just saying something to their LLM that's a question or asking you to do something.
00:08:39.600 -->
00:08:41.038So, you know, hey, I love the pesto.
00:08:41.038 -->
00:08:42.480I love my running shoes.
00:08:42.879 -->
00:08:42.879Yeah.
00:08:42.879 -->
00:08:50.480I guess maybe maybe pesto and running shoes aren't the best expression, whereas expression might be like, I'm really and overwhelmed.
00:08:50.480 -->
00:08:55.600And sometimes people just say that to their LLM in lieu of it to anyone else.
00:08:55.918 -->
00:08:56.720And I think this is pretty unique.
00:08:56.720 -->
00:08:59.759Like, I think this is not something that we're all doing with Google.
00:08:59.759 -->
00:09:02.558It's not something we're used to doing with our computer.
00:09:02.558 -->
00:09:05.038So I think this is kind of new.
00:09:05.519 -->
00:09:14.720And listeners, just to give you a bit of an a behind-the-curtain look into the podcast and Hughes and my's relationship, is I'm very fast and loose with how I think about my data.
00:09:14.720 -->
00:09:19.200And so Hugh and I had like a big debate the other day of what's the big deal with these LLMs?
00:09:19.200 -->
00:09:20.798Like, I'm giving all my data away.
00:09:20.798 -->
00:09:26.000If there's one thing I took from episode one of this season, I'm giving my data to everyone anyway.
00:09:26.000 -->
00:09:45.840And this expressing point is the one where you sort of won me round in our debate because that isn't anything that's Google or in WhatsApp or in your metadata from Google Maps is this whole like emotional piece, which is I don't know if it expected with these LLMs, but like it's certainly taken me by surprise.
00:09:46.639 -->
00:09:59.278In the OpenAI paper, they dug even deeper, Hannah, and they figured out that about 1.9%, so just under 2% of all are sort of relationships, personal reflection that people are having.
00:09:59.278 -->
00:10:02.639And about half a percent is role play and companionship.
00:10:02.639 -->
00:10:06.798So people kind of, I guess, trying to have a relationship their LLM.
00:10:07.600 -->
00:10:18.240Okay, well, I have a separate stat that there are two and billion prompts a day to Chat GPT alone of a user set of two and three hundred million users.
00:10:18.240 -->
00:10:19.759So what did you say?
00:10:19.759 -->
00:10:34.240Like one and a half percent sounds fairly low, but in absolute terms, that is a lot of people, or maybe a smaller number of using it a lot for this relational emotional support which is that is bizarre.
00:10:35.120 -->
00:10:46.158With those numbers too, actually, in this September paper OpenAI last year, the latest number they had was 700 users per week, which maybe is 300 million a day, maybe too far apart.
00:10:46.158 -->
00:10:49.120And that graph was going up to the right and almost vertical.
00:10:49.120 -->
00:11:01.038So I think we could probably safely assume now that certainly more than a billion users a week using these and probably that 2.5 billion is probably more like three a half, four billion today.
00:11:01.038 -->
00:11:04.158And heaven knows what it will be by the end of uh this
00:11:04.960 -->
00:11:05.278Okay, wow.
00:11:05.278 -->
00:11:08.399So quick follow-up on this whole expressing use case.
00:11:08.399 -->
00:11:14.960You've got two daughters who are in their early 20s, so like in the younger generation, very tech savvy.
00:11:14.960 -->
00:11:17.120Do you think that they use it?
00:11:17.120 -->
00:11:24.879Not necessarily to say they're using it for the expressing case, but do you think that they're using it differently to how you would use it or I would use it?
00:11:24.879 -->
00:11:29.840Because actually, between me, you, and your daughters, we span three different generations, right?
00:11:30.240 -->
00:11:38.158Well, first of all, here's the big stat uh almost half messages to Chat GPT, half of the prompts come from people are under 26.
00:11:38.158 -->
00:11:39.678So that's the biggest difference.
00:11:40.240 -->
00:11:42.879Half of those prompts are coming from people under the age of 26.
00:11:42.879 -->
00:11:43.840Wow.
00:11:44.558 -->
00:11:49.038Yep, which just shows you that there's very, very heavy usage, for example.
00:11:49.038 -->
00:11:59.360So lots and lots of people using it for homework, writing, social media posts, messages, emails, if they're sending any.
00:11:59.360 -->
00:12:04.960It's open all over their screens, whether it's the smartphone or not, uh, and it's the copilots through life.
00:12:04.960 -->
00:12:12.879Because the other really interesting stat that came out of paper is that 73% of the usage of Chat GPT is non-work.
00:12:12.879 -->
00:12:20.798So I think the myth out there is that it's a work tool, it's a productivity tool, but the reality is in fact that it's three quarters non-work.
00:12:21.918 -->
00:12:33.038That is interesting because the whole rhetoric around the power and the disruption of AI is about how it's changing the way that we work.
00:12:33.038 -->
00:12:43.600And whilst it certainly is, and I think I mentioned in a episode, like my husband's work have now implemented an AI where if you don't use AI every week, they call you up on it.
00:12:43.600 -->
00:12:44.720So there's some of that.
00:12:44.720 -->
00:12:54.320That feels a bit contrived though, whereas actually people are reaching for it to solve everyday problems that aren't work I think that certainly fits my pattern of use.
00:12:54.320 -->
00:13:02.639I'm definitely using it more for recipes, workouts, general I'm using it to explain things to me more than I'm using it summarize documents.
00:13:02.639 -->
00:13:11.759I actually think in that when I asked for the summary, the that I mentioned up front in the episode, it actually called out you're not using this for automation or summarizing.
00:13:11.759 -->
00:13:13.678You're using it like more conversationally.
00:13:14.960 -->
00:13:30.558And I guess, look, you know, if half the prompts are coming from people under 26, then I guess it's not all that that it's more non-work use than work use, given likely that population is still studying, doesn't yet have its first full-time job, and so on.
00:13:30.558 -->
00:13:35.360So I think those things are correlated, but you know, it's a personal productivity tool used by young people.
00:13:35.840 -->
00:13:37.360Okay, so follow-up question.
00:13:37.360 -->
00:13:54.158If it is much more personal queries and in some cases like expression, is it worrying that we're pumping all of this language models, which presumably they're collecting and for various different things?
00:13:54.158 -->
00:13:55.120Like, is that a concern?
00:13:55.120 -->
00:13:59.600Because I care less, if I'm honest, about what company documents I'm putting in.
00:13:59.600 -->
00:14:09.200By the way, I am putting it into the enterprise Chat GPT, so licensed with all the right things, but that's less of a concern to me than some of the more personal stuff that I might be through with ChatGPT.
00:14:10.158 -->
00:14:12.080Yeah, and maybe let's just talk about that a little bit.
00:14:12.080 -->
00:14:17.600What happens when you actually do type into Chat GPT, Gemini, whatever it is.
00:14:17.600 -->
00:14:30.558So first thing is you're in the UK right now, I'm in There's absolutely no guarantee that the data that you're in is actually stored in Australia or the UK, respectively, right?
00:14:30.558 -->
00:14:31.678So that's that's not guaranteed.
00:14:31.678 -->
00:14:37.120Quite likely that it is, given, you know, they're both big for these companies.
00:14:37.120 -->
00:14:45.200But, you know, it could just as easily be stored in Europe in your case, um, or it could be stored on the west of the US in my case.
00:14:45.200 -->
00:14:51.918So the data is potentially leaving the country and somewhere else, and you have no control over that.
00:14:52.399 -->
00:15:03.360Okay, so on that basis, Hugh, thinking about data collection storage, what data policies or data protection laws is my data in the UK subject to?
00:15:03.360 -->
00:15:08.720Is that based on where the data is being stored or is that on where it's being collected?
00:15:09.440 -->
00:15:09.759Yeah, both.
00:15:09.759 -->
00:15:10.558Both, Hannah.
00:15:10.558 -->
00:15:12.798So let's imagine we'll make up a fictional case, right?
00:15:12.798 -->
00:15:22.240So let's imagine that your data that you're typing in is in the EU somewhere, and obviously the UK is famously not of the EU anymore.
00:15:22.240 -->
00:15:28.480So, first of all, the company is offering the service in UK to you, and so they have to comply with UK laws.
00:15:28.480 -->
00:15:38.879So there's going to be some privacy laws in the UK, and the company is operating their service in the UK, they have to make their best effort to comply with UK privacy laws, they are.
00:15:38.879 -->
00:15:39.200Right.
00:15:39.200 -->
00:15:40.158So that's the first thing.
00:15:40.158 -->
00:15:50.320The second thing is because the data is fictionally the EU, they also have to comply with the storage rules around data that's stored in the EU.
00:15:50.320 -->
00:16:01.678So whatever the privacy limitations are, the anonymization are, the retention limitations are, they have to be to your data because it's stored in the EU.
00:16:01.678 -->
00:16:03.600And it could be subpoenaed in the EU.
00:16:03.600 -->
00:16:18.000So if uh if somebody wants to access your data, they they could force the company to extract it from their EU center and give it to them, even though you never knew your data was stored in the EU.
00:16:18.000 -->
00:16:20.000So two things are applying at once.
00:16:20.639 -->
00:16:25.600It's worth then knowing what different products are coming of which countries, right?
00:16:25.600 -->
00:16:31.918Like I know this came up when Deep Seek that AI model that out of China, that must have been sort of the beginning of year.
00:16:31.918 -->
00:16:37.038And we got some warnings at work about what to use it for from like an IP perspective.
00:16:37.038 -->
00:16:41.840But presumably, countries like China, look, we know they have a different approach to data.
00:16:41.840 -->
00:16:49.519And so if you're using a model that comes out of China or is related to China, then it will be subject to a different on data protection.
00:16:50.158 -->
00:16:50.720Yeah, that's right.
00:16:50.720 -->
00:16:51.200That's right.
00:16:51.200 -->
00:16:54.960And obviously, you know, different countries all over the have different approaches to these things.
00:16:55.200 -->
00:16:56.639And you know, and some might feel familiar.
00:16:56.639 -->
00:17:02.879So if you're in the UK, the EU privacy laws will feel pretty familiar, and some might feel a lot less familiar.
00:17:02.879 -->
00:17:05.118So let's continue our fictional example here.
00:17:05.118 -->
00:17:25.038So let's imagine that you're in the UK, you type a prompt, data's stored in the EU, but behind the scenes, the folks at Apenai want to, you know, they want to inspect some prompts, and by chance they inspect your prompt and they It could be evaluated in Poland, it could be evaluated in it could be evaluated in the UAE.
00:17:25.038 -->
00:17:41.038Um, it just depends on where they've hired that contingent to evaluate the prompts and provide them with feedbacks on So your data could simultaneously be subject to the rules of three different places or potentially even more.
00:17:41.598 -->
00:18:05.038Well, I was gonna say if I'm in the US or a different country outside of the EU for work, or but let's say when I was in with you, I'm a UK user, which it must know it's processing data, I guess, based on where I'm launching the query, and could be pitched all around the world for the human in the loop validation if it gets called out for that.
00:18:05.038 -->
00:18:11.759And as we've talked about in previous episodes, that'll be like a low-cost workforce that could be out of any of the that you just mentioned.
00:18:11.759 -->
00:18:27.680So there's a lot of complexity in how it's being And the tr the reality is we don't know, and they can do all and it's lawful because as long as they're not breaking any they can, you know, I'm handing my data and it's up to them use it within reason, how they need to or see fit.
00:18:28.160 -->
00:18:28.480Yeah, absolutely.
00:18:28.480 -->
00:18:34.640And and even with their best efforts, it can be very complex for them to manage these data sets across the world.
00:18:34.640 -->
00:18:42.000And that's why their legal folks get paid an enormous of money and uh why they need very, very smart people working in their companies, because this is very, very complex.
00:18:42.000 -->
00:18:49.598And the technology actually makes it even more complex, So, you know, back in the old days when I was working at we used to store three copies of everything.
00:18:49.598 -->
00:18:55.759And the reason we did that is because we were storing data very cheap computers that are very, very unreliable.
00:18:55.759 -->
00:18:59.038And so you want to have three copies of all of the data.
00:18:59.038 -->
00:19:03.920You want to put those in smart places just in case you lose one of the copies because the computers are unreliable.
00:19:03.920 -->
00:19:08.798And then if you lose one of the copies, you furiously make copy so that you've got three again.
00:19:08.798 -->
00:19:11.440And that gives you what's called 11 nines of reliability.
00:19:11.440 -->
00:19:17.519So you have 99.99999 up to nine nines after the decimal of reliability.
00:19:17.519 -->
00:19:21.759So you have almost no chance of losing any data ever if have three copies.
00:19:21.759 -->
00:19:31.200Folks like OpenAI and Enthropic today are well aware of And so they are also making sure that they have multiple of the data, and they're pretty smart about where they store them.
00:19:31.200 -->
00:19:38.318They're more sophisticated than we were back in So we used to just make sure that we had them in different of the data center, maybe across two data centers.
00:19:38.318 -->
00:19:42.558They're really, really smart about different geographies, data centers.
00:19:42.558 -->
00:19:44.240And so there's even more complexity.
00:19:44.240 -->
00:19:58.960So my hunch is they don't just have three copies of most of the data, they probably have four, five, or six copies, and those copies are deliberately going to be in very places in case they lose a whole data center or maybe even a whole region of data centers.
00:19:58.960 -->
00:20:00.558So your data is everywhere, Hannah.
00:20:01.200 -->
00:20:12.400And when you think about the complexity of the different they need to be preparing for, as you say, like it's really because we're such a global world these days.
00:20:12.400 -->
00:20:27.519You're downloading an app or you're primarily using an app in one location, your home location, the app's from a different and then you're traveling around and you're subject to sort of all those jurisdictions and there are specific requirements of companies that collect data.
00:20:27.519 -->
00:20:30.318So that is sounds like an enormous headache, to be honest with you.
00:20:30.640 -->
00:20:31.358Yeah, enormous headache.
00:20:31.358 -->
00:20:32.160Enormous headache.
00:20:32.160 -->
00:20:35.519And you know, you've got to remember these companies are sort of startup growth kinds of companies.
00:20:35.519 -->
00:20:40.000So they're gonna be doing their best, but it's also gonna be quite chaotic inside these companies.
00:20:40.000 -->
00:20:43.680And so they're definitely gonna be moving fast and breaking things, as they say.
00:20:43.680 -->
00:20:49.358And, you know, they're gonna be a lot of tension with the teams and a lot of tensions about where they launch and they do.
00:20:49.358 -->
00:20:58.798Um, so it's gonna be a very, very complex world when it to trying to make sure that they're complying with all of laws and making sure that the data's stored where they think it is, and uh, you know, so on and so forth.
00:20:58.798 -->
00:21:08.798But definitely, once you type your data in and you press that ender key, you know, you're setting your data free, many different jurisdictions, many different data right across the world.
00:21:08.798 -->
00:21:10.960And, you know, you've kind of lost control of that.
00:21:11.440 -->
00:21:15.200One last point on data before I move us along, and I don't there's a definitive answer to it.
00:21:15.200 -->
00:21:24.720It's just an observation of mine is that with things like LLMs, no one has more resource or better infrastructure to progress than these companies, right?
00:21:24.720 -->
00:21:27.440They're commercial companies, they're getting tons of at the minute.
00:21:27.440 -->
00:21:34.798On the flip side, like government agencies, in particular legal departments of government agencies, are not known efficiency.
00:21:34.798 -->
00:21:38.240You know, they're very thorough, they might not be super resourced.
00:21:38.240 -->
00:21:51.440And there's a weird sort of disparity there because I just see a world in which it all keeps up to the pace of the tech And I don't know whether that's a problem or not, but it's it's definitely there's definitely a contrast there.
00:21:51.838 -->
00:21:56.400I think it's super problematic, Hannah, with all due to smart lawyers who work in governments.
00:21:56.400 -->
00:21:58.720These are not AI professionals, right?
00:21:58.720 -->
00:21:59.519They don't understand the tech.
00:21:59.519 -->
00:22:02.078Technology in depth, they may not even be listeners to this podcast.
00:22:03.358 -->
00:22:07.200Well, but also the the most expert people are the ones working in the companies, right?
00:22:07.200 -->
00:22:24.798So even when it comes to maybe we'll get into it in more later, but the the recent like Pentagon versus Anthropic, when there was debate around the use case of LLMs in autonomous and the government wanted the final say on whether or not the LLMs were safe to use.
00:22:24.798 -->
00:22:35.118No one knows the LLMs better than the people who built and with the LLMs, but they're all in the companies running the They're not in all of the sort of adjacent medical boards and government agencies.
00:22:35.118 -->
00:22:38.480So there's a pace disparity there that's problematic.
00:22:39.278 -->
00:22:41.680And look, it brings me to one thing that I really dislike Australia.
00:22:41.680 -->
00:22:46.880You know, I've I've spent the last nine years, I think, now back in Australia after my long stint in the US.
00:22:46.880 -->
00:22:55.278And one of the things that infuriates me most about is that we try and legislate what these companies can do and stop them innovating.
00:22:55.278 -->
00:23:01.519I much prefer the US climate where the companies are a bit more freer to do the things that they want to do.
00:23:01.519 -->
00:23:07.440And then if they do things that seem outside the bounds then people sue people.
00:23:07.440 -->
00:23:19.440And so, you know, somebody will sue somebody else and send a legal precedent that says that's where the boundary And so it's it's a lot more free and loose in the US, and you know, it's more of a reaction to what happens.
00:23:19.440 -->
00:23:29.680Whereas I feel like in Australia, we'll have the Australian AI policy bill, and we'll we'll debate this thing and release it, and we'll constrain all innovation before the even happened.
00:23:29.680 -->
00:23:32.078And it's incredibly infuriating, Hannah.
00:23:32.640 -->
00:23:42.720It's an interesting debate, and one other facet of it that I about quite a lot is that this AI revolution is taking place a Trump administration in the US.
00:23:42.720 -->
00:23:49.278And I'm wondering, like, what the heck kind of bearing is gonna have on where we land in the future on all this stuff?
00:23:49.278 -->
00:23:58.558Because it's certainly the most unique administration of my Yeah, we're having this huge societal and technological in the way we live our lives.
00:23:58.558 -->
00:24:03.278And the US is normally like a real player in shaping how these things unfold.
00:24:03.278 -->
00:24:07.598And it's just quite a unique world we find ourselves in on front.
00:24:08.000 -->
00:24:08.640Yeah, yeah, absolutely.
00:24:08.640 -->
00:24:13.598And look, it's a fairly pro-tech administration, I guess, is one thing that you might observe from a distance.
00:24:13.598 -->
00:24:16.720Um, so it certainly will be interesting to see how it how it plays out.
00:24:17.118 -->
00:24:28.960Okay, so let's get back to the sort of nuts and bolts of So we've putting all this data in, it's being processed under some sort of government jurisdiction, bit of complexity But the actual ways that the data are being used.
00:24:28.960 -->
00:24:51.038So if I call back to our episode around like our phones to us, and we got talking about like metadata and how process data, it's obviously super valuable for But it's generally processed at an aggregate level, Like it's not likely that someone's going in and reading my prompts with Chat GPT unless it's like a human in the loop test.
00:24:51.038 -->
00:24:54.318But I assume those are pretty low as a percentage of
00:24:55.200 -->
00:24:57.038You got billions of these prompts per day, Hannah.
00:24:57.038 -->
00:25:10.400So it's certainly not the case that any significant fraction of them are ever looked at by humans, but certainly some You've got this incredibly small probability that is actually looked at by a human.
00:25:10.400 -->
00:25:18.400And the reason that somebody at OpenAI or at Anthropic know, at Google or Microsoft would look at this prompt improve the product, right?
00:25:18.400 -->
00:25:28.318So they would open up your prompt and they'd make an You know, does it look like this user was happy with the Does it look like the LLM successfully answered the question?
00:25:28.318 -->
00:25:32.160So just asking simple questions around the performance of tool.
00:25:32.160 -->
00:25:37.118And that's used in, you know, all kinds of different ways the companies are improving the products.
00:25:37.118 -->
00:25:40.400But the chances of that happening is very, very small.
00:25:40.400 -->
00:25:42.720And you can actually control it.
00:25:42.720 -->
00:25:55.519So if you do go into ChatGPT and you go into your you go down to data controls, if you do go into data you'll find there's a little slider for an option that's improve the model for everyone.
00:25:55.519 -->
00:25:57.278And it's turned on by default.
00:25:57.278 -->
00:26:00.558And what that means is basically two things, Hannah.
00:26:00.558 -->
00:26:05.440So the first thing is that your content can be used to future models.
00:26:05.440 -->
00:26:12.640And also it means that humans can inspect your data for quality improvement process.
00:26:12.640 -->
00:26:16.880So if you kind of want to opt out of that, you can just turn that toggle off.
00:26:16.880 -->
00:26:23.838And then your data won't be used in any future training and it won't be looked at by individual humans.
00:26:24.720 -->
00:26:26.400Do you think that will always be an option?
00:26:26.400 -->
00:26:34.720Like I see some websites, if you want anything other than the standard cookies, which is a similar sort of like personal data point.
00:26:34.720 -->
00:26:37.680You have to use the paid for version if you want to turn that stuff off.
00:26:37.680 -->
00:26:39.759Do you think that is where this will eventually head?
00:26:39.759 -->
00:26:44.400Which is like if you want the privilege of us not using the for training, you're gonna have to pay a subscription.
00:26:45.118 -->
00:26:45.838No, I don't think so.
00:26:45.838 -->
00:26:53.759I think I think what happens with these advanced options, And remember, I'm I'm well down in the settings here, For most of our curious listeners, this is going to be new right?
00:26:53.759 -->
00:26:59.038That you can actually go into settings, data controls, and off, improve the model for everyone.
00:26:59.038 -->
00:27:00.960Most of our listeners are gonna go, oh, that's cool.
00:27:00.960 -->
00:27:01.759I might go do that.
00:27:01.759 -->
00:27:04.480With these advanced options, nobody ever turns them off.
00:27:04.480 -->
00:27:04.880Right.
00:27:04.880 -->
00:27:11.519So you would find that probably 0.1 of a percent, 0.2 of a of users today have actually turned this off.
00:27:11.519 -->
00:27:19.358And uh, even with us releasing a podcast that explains should do it, it's really not gonna change how people access these advanced options.
00:27:19.358 -->
00:27:21.200So I just don't think it's an issue for them.
00:27:21.200 -->
00:27:28.558As long as the the toggle is turned on by default, they're be happy campers because nobody will actually go into an option and turn it off.
00:27:29.038 -->
00:27:47.838And there's also other things that you can toggle, it's related, but my husband was telling me that you can change the like level of enthusiasm and the tone or like level of So he recently turned his to be like meaner because there's whole that well, maybe we should talk about this for a second.
00:27:47.838 -->
00:27:56.720There's this whole phenomenon that like LLMs can be in their nature, which I presume gets people to use them more because it's nice when someone agrees with you.
00:27:56.720 -->
00:28:06.318But I did read that there's like a it can be a bit problematic with conspiracy theorists who just get like validation on view that the world is flat from Chat GPT.
00:28:06.318 -->
00:28:08.240That's like, hey, that's a great point.
00:28:08.240 -->
00:28:10.640And there's you know plenty of reason to believe it's true.
00:28:10.640 -->
00:28:15.358So he made his a bit more cynical, but you can make it like chatty, you can make a bit more positive.
00:28:16.160 -->
00:28:16.720Yeah, you can.
00:28:16.720 -->
00:28:23.920And I think this is a reaction by the folks at OpenAI to if you like, because Claude has always been a little bit your friend, a little bit more chatty.
00:28:23.920 -->
00:28:35.440And so I think the ChatGPT folks have made it possible control the sort of style and tone of ChatGPT to allow people to make it be a little bit more Claude-like or perhaps pick a pick a different personality.
00:28:35.440 -->
00:28:44.160But if our listeners want to follow along, you can go into GPT, into the settings, and into personalization, and you'll see an option called bass style and tone.
00:28:45.118 -->
00:28:46.798can indeed pick different kinds of personalities.
00:28:46.798 -->
00:28:50.960I've got mine set to candid because I just wanted to get the point.
00:28:50.960 -->
00:28:57.759Um, but you can choose professional, friendly, candid, efficient, nerdy, and my favorite, cynical.
00:28:58.640 -->
00:28:58.640Cynical.
00:28:58.640 -->
00:29:00.078That sounds very British.
00:29:00.078 -->
00:29:01.598I think the quirky would be annoying.
00:29:01.598 -->
00:29:14.720I might play around with it because yeah, now that you've me to that in the app, there's bass style and tone, and these characteristics, including like how much it uses and then whether it's like warm, enthusiastic.
00:29:14.720 -->
00:29:16.480Interesting.
00:29:17.278 -->
00:29:20.640Yep, so you can make yours less enthusiastic, less of a of a cheerleader, Hannah.
00:29:20.640 -->
00:29:26.720I think when it comes back to conspiracy theories, Hannah, know, we we talked about this a lot back in episode seven of season one.
00:29:26.720 -->
00:29:31.358We have to just remember that these are probabilistic, you monkeys, if you like.
00:29:31.358 -->
00:29:40.480And so ChatGBT, it's got this huge context window of all different words and tokens, and then it's generating output that follows those, right?
00:29:40.480 -->
00:29:50.480So if you're having a long conversation about a conspiracy about the earth being flat or whatever it is, it's going generate sentences that follow on nicely from the context of what you've been talking about.
00:29:50.480 -->
00:29:55.680So, of course, surprise, surprise, it will agree with you and reinforce the kinds of conversation that it's having.
00:29:55.680 -->
00:29:57.598I mean, that's just how this technology works.
00:29:58.160 -->
00:30:01.200So it's creating even more echo chambers for folks.
00:30:01.200 -->
00:30:01.680Yeah.
00:30:02.078 -->
00:30:06.798It's producing plausible output given the context that it That's all these LLMs do.
00:30:07.200 -->
00:30:13.759Yeah, and it's dressed up so cleverly to feel like you're a conversation with someone to whom you want to divulge it.
00:30:13.759 -->
00:30:15.598Sounds like deep personal information.
00:30:15.598 -->
00:30:16.960But yeah, you're right.
00:30:16.960 -->
00:30:24.240This is literally just a probabilistic model giving you the likely satisfactory response as a result of the prompt and that is it.
00:30:24.558 -->
00:30:27.278Yeah, and it's incredibly impressive because it's trained on trillions of words.
00:30:27.278 -->
00:30:29.358It cost a blockbuster movie to train.
00:30:29.358 -->
00:30:32.078It is an impressive probabilistic machine.
00:30:32.078 -->
00:30:41.358And so the context that it produces is incredibly relevant the conversations that you're having or the task that doing, but it is a probabilistic text generator.
00:30:41.358 -->
00:30:46.078And uh, we're gonna have a really good interview coming up in the season, Hannah, about the future of AI.
00:30:46.078 -->
00:30:55.440And I'm gonna get into it with our guest who I want to know into a spoiler yet, but we're gonna get into it with our and talk about the future of AI and you know, is it possible that these machines will ever become intelligent?
00:30:55.440 -->
00:30:57.519But they're certainly not intelligent today.
00:30:58.000 -->
00:31:03.920Yeah, they're just dressed up to feel really intelligent, which is the whole that is the secret sauce, right?
00:31:03.920 -->
00:31:08.480Suddenly AI has come to the fore, and that's the reason because it feels like we're talking to someone.
00:31:08.480 -->
00:31:17.598And I guess that harps back to the point we made, which is most people are reaching to this for majority non-work use cases it's like a pocket digital life coach.
00:31:17.598 -->
00:31:19.200I think I saw it described as.
00:31:20.160 -->
00:31:20.640probably pretty fair.
00:31:20.640 -->
00:31:33.920Like all the all the things we discussed, whether it's, you workout advice or recipes or something more interesting than it's all under that umbrella of life coach, career, you sometimes relationship advice.
00:31:34.640 -->
00:31:35.118Yeah, absolutely.
00:31:35.118 -->
00:31:35.920Absolutely.
00:31:36.798 -->
00:31:38.720Can we talk a little bit more about training?
00:31:38.720 -->
00:31:48.960I've actually just opted out of training while we've been Um, is that something we should be worried about or inverse, that something we should be open to?
00:31:48.960 -->
00:31:51.838Because I think it says in the app, do you want to improve it for everyone?
00:31:51.838 -->
00:31:54.480And you're like, oh, well, I do kind of want to improve it for everyone.
00:31:54.480 -->
00:31:55.920But what does that mean in practice?
00:31:56.558 -->
00:32:05.920Yeah, so what that means in practice is that in future runs, if you are, if you've opted in, your data can be in the training process.
00:32:05.920 -->
00:32:13.200But I think what our listeners have to remember is it's with billions of other prompts typed in per day.
00:32:13.200 -->
00:32:16.798And so you are, you know, a drop in an ocean isn't even fair, right?
00:32:16.798 -->
00:32:18.880You are less than a drop inside an ocean.
00:32:18.880 -->
00:32:30.318And so very literally, you know, what will happen is that influence in a very minute way the style and the of a word appearing in a future model.
00:32:30.318 -->
00:32:36.078Your text is never going to be verbatim reproduced by the in a future model.
00:32:36.078 -->
00:32:38.960So you might say, Well, I really want to protect my privacy.
00:32:38.960 -->
00:32:42.400I don't want them to use it in training, and you might choose to turn that off.
00:32:42.400 -->
00:32:54.640But I think by leaving it on, all you're doing is becoming of the masses that influences the probability of a word generated and perhaps, you know, the probability of a tone of phrase coming out of the LLM.
00:32:54.640 -->
00:32:58.000So I think in practice, it probably doesn't really matter.
00:32:58.558 -->
00:33:03.598That's a bold statement, but we're going to make it on this It doesn't really matter on an individual level.
00:33:04.160 -->
00:33:05.200No, no, I don't think so.
00:33:05.200 -->
00:33:18.400But I would say, you know, because humans can inspect and because you're putting your data in an unknown and because your data's moving around the world, you should be a little thoughtful about what data you these systems.
00:33:18.400 -->
00:33:26.160So personally, I would not upload my medical records and have them stored randomly in some country, randomly copied the world.
00:33:26.160 -->
00:33:33.278And I'm I'm not sure that I'd want any probability of a human opening up and having a look at a PDF of my personal medical records.
00:33:33.278 -->
00:33:39.598So I think we should all be thinking about what data we're And I think if we're not comfortable with that, then just upload the data.
00:33:40.000 -->
00:33:41.519Yeah, so common sense use.
00:33:41.519 -->
00:33:43.920It's probably never gonna get looked at.
00:33:43.920 -->
00:33:46.240There's a small probability that it will.
00:33:46.240 -->
00:33:51.519Don't share anything too overtly personal because you just do you shouldn't do that anyway.
00:33:52.160 -->
00:33:52.558Yeah, absolutely.
00:33:52.558 -->
00:33:54.318So just using common sense.
00:33:54.318 -->
00:34:00.240But in practice, you know, your personal details aren't to appear out of a model in the future for somebody else.
00:34:00.240 -->
00:34:02.640You just don't have that influence in the training process.
00:34:03.200 -->
00:34:03.519Fair enough.
00:34:03.519 -->
00:34:10.480I do think people will get a kick though out of um now that tipped everyone off how to look at their memories, it'll be they're a Spotify rap.
00:34:10.480 -->
00:34:14.800They'll go into their chat chat GPD and see um see what it of them.
00:34:15.119 -->
00:34:19.518So maybe, maybe homework for you, Hannah, is uh before our wrap episode, a bit more expressing.
00:34:19.518 -->
00:34:21.440And uh you can report back.
00:34:21.440 -->
00:34:26.800We'll we'll chat about how our LLM uh use changed based on uh observations of LLM use.
00:34:26.800 -->
00:34:27.360What do you think?
00:34:27.679 -->
00:34:28.559Yep, that's totally fair.
00:34:28.559 -->
00:34:34.960And this whole expressing point, like I wonder if I should be using ChatGPT more as a coach.
00:34:34.960 -->
00:34:37.119I don't know whether it's a good thing or a bad thing.
00:34:37.119 -->
00:34:46.239It was certainly an interesting thing that that's a a that's bubbling up in sort of over 10% of prompts, but um not one I've explored in depth just yet.
00:34:46.239 -->
00:34:49.760Okay, guys, this has been the Tech Overflow Podcast.
00:34:49.760 -->
00:35:02.159If you've liked what you've heard, please, please, please like, subscribe, leave us a review on Spotify, Apple Podcasts, with your friends, family, anyone who you think might be about how tech works.
00:35:02.559 -->
00:35:05.280Yeah, and if you'd like to learn more about the show, you follow us on our socials.
00:35:05.280 -->
00:35:15.840We are on LinkedIn, X, Instagram, TikTok, and YouTube And of course, we've got our own website, So we'll see you next time, Hannah.
00:35:15.840 -->
00:35:16.719Take care, guys.
00:35:16.719 -->
00:35:17.679Take care, bye.
