WEBVTT
00:00:03.439 --> 00:00:06.240
Jesse Chambers is a travel expert.
00:00:08.880 --> 00:00:11.599
Quoted across British newspapers.
00:00:14.080 --> 00:00:22.559
She gives advice on everything from cruise ships to long-haul flights.
00:00:22.719 --> 00:00:24.000
There's just one problem.
00:00:24.559 --> 00:00:26.879
Justice Chambers doesn't exist.
00:00:30.640 --> 00:00:32.960
Welcome to the Machine Dialogues.
00:00:33.119 --> 00:00:36.000
Conversations about AI, systems, and the self.
00:00:36.159 --> 00:00:44.640
Exploring what happens to identity in an increasingly synthetic world as we evolve into machinable and unmachinable selves.
00:00:49.600 --> 00:00:55.439
Today's guest is technology journalist Rob War, a prominent British tech journalist.
00:00:55.679 --> 00:01:00.399
Rob has spent years reporting on the evolution of the internet and emerging technologies.
00:01:00.560 --> 00:01:08.400
He's written for titles including the Daily Telegraph, Daily Mail, and Metro, and previously served as science editor at the Mail Online.
00:01:08.799 --> 00:01:15.439
He also launched the podcast networker Associated Newspapers, overseeing shows across news, politics, and culture.
00:01:16.719 --> 00:01:20.319
Recently, Rob stumbled onto something unusual.
00:01:20.560 --> 00:01:24.640
A series of media experts appearing in newspapers across Britain.
00:01:24.879 --> 00:01:28.640
Travel specialists, palace insiders, lifestyle commentators.
00:01:28.879 --> 00:01:31.680
Turned out to be synthetic biographies.
00:01:31.920 --> 00:01:38.560
These were experts manufactured to give quotes to journalists, quietly inserted into the news ecosystem.
00:01:38.879 --> 00:01:50.319
In this episode, we talk about the rise of fake experts, how synthetic authority is slipping into mainstream media, and what happens when even expertise itself becomes machine generated.
00:01:50.480 --> 00:01:58.159
Because if expertise can be generated, identity can be fabricated, and then authority can be automated.
00:01:58.560 --> 00:01:59.920
That's what we're exploring.
00:02:00.079 --> 00:02:02.640
This week on machine dialogues.
00:02:07.760 --> 00:02:08.639
Rob War.
00:02:09.039 --> 00:02:11.120
Welcome so Machine Dialogues.
00:02:11.280 --> 00:02:12.400
You are the first.
00:02:12.879 --> 00:02:21.599
And I really wanted to talk to you because obviously the subject of machine dialogues is around the machinable and the unmachinable parts of the self.
00:02:21.759 --> 00:02:28.960
And you in particular have been doing this groundbreaking work on what I would call identity slop.
00:02:30.080 --> 00:02:31.199
There's a lot of it about.
00:02:31.439 --> 00:02:31.680
Yeah.
00:02:31.840 --> 00:02:49.360
I wonder, especially for the listeners, if you could take us back to the beginning and tell us what you uncovered when you were looking at these experts that were appearing in really quite, you know, quality press outlets with their varied opinions.
00:02:50.240 --> 00:02:50.639
Sure.
00:02:50.879 --> 00:02:58.960
Well, I mean, it was a weird one because the first time it happened, it happened a year before the piece, first piece I wrote for Press Gazette.
00:02:59.039 --> 00:03:04.879
I was doing some freelance work for a marketing agency, and we got approached by somebody to do a guest blog.
00:03:05.120 --> 00:03:07.759
And I looked at the person and I thought, this is really strange.
00:03:07.919 --> 00:03:09.439
This person doesn't exist.
00:03:10.240 --> 00:03:13.360
And the other people on the site were just going, No, what do you mean?
00:03:13.520 --> 00:03:17.680
What why is the and I said, really, take it from me, this person doesn't exist.
00:03:17.840 --> 00:03:19.280
I forgot all about it.
00:03:19.439 --> 00:03:30.240
And then a year later, I was using Responsours, which is a response service for journalists, where if you need somebody to talk about something, you send a request for a specific kind of expertise.
00:03:30.400 --> 00:03:37.039
And I needed a psychiatrist to talk about the psychological impact of identity fraud.
00:03:37.199 --> 00:03:47.120
And I sent out a request and I got a response from this psychiatrist, an Oxford-educated psychiatrist, no less, who called herself Barbara Santini.
00:03:47.199 --> 00:03:50.240
And the thing was that the copy seemed a little strange.
00:03:50.400 --> 00:04:05.199
It had things that I think we're all very familiar with now as a result of ChatGPT, weird sort of capitalized subheads inserted into the copy for no reason and stuff, which often are quite hallmarks of LLM writing.
00:04:05.360 --> 00:04:07.039
And I was like, this is a bit odd.
00:04:07.120 --> 00:04:12.879
But really, the reason I sort of looked into it was because I was writing for the telegraph and she worked for a sex shop.
00:04:12.960 --> 00:04:15.520
And I thought, I can't put a sex shop in the telegraph.
00:04:15.680 --> 00:04:20.079
You know, some old Tory Kerner will have a heart attack into his cornflakes.
00:04:20.319 --> 00:04:28.000
And uh so I sort of thought, oh well, I'll just look up what her other jobs are, and I'll just list something else.
00:04:28.160 --> 00:04:32.480
And then when I looked, I realized that she wasn't real, there was just no other jobs.
00:04:32.639 --> 00:04:39.759
The only thing that she existed and did was speaking to newspapers, but she'd been hugely successful.
00:04:39.920 --> 00:04:50.639
She, in fact, she'd been in newspapers 120 times, and that included all the big sort of popular ones, you know, Mail, Express, Mirror, Sun, dozens of times.
00:04:50.800 --> 00:04:57.360
And she'd even penetrated into the BBC, The Guardian, International Press, like a New York Post.
00:04:57.519 --> 00:05:06.160
And um, when we confronted the organization behind her, we got a series of extremely convincing sounding legal threats.
00:05:06.240 --> 00:05:12.240
Because uh, I won't say his name, but the the guy who's actually behind this is uh he's a qualified solicitor.
00:05:12.399 --> 00:05:14.720
So it's like the legal threats were.
00:05:15.120 --> 00:05:16.079
Is he though?
00:05:17.199 --> 00:05:24.800
They were there were they were high there were highly convincing legal threats because you get legal threats a lot of the time, but then occasionally you get ones where you're like, okay, this is serious.
00:05:24.959 --> 00:05:34.959
And uh anyway, we published the piece, and the legal threat said that they would sue immediately if we said that this person didn't exist.
00:05:35.199 --> 00:05:39.519
We said it, and then the legal suit didn't happen.
00:05:39.680 --> 00:05:43.759
And over the coming months, I began to realize that she was far from a loan.
00:05:44.079 --> 00:05:47.120
There are a huge number of fake experts.
00:05:47.360 --> 00:05:55.199
Last time we counted it up, there was more than 50 and more than a thousand articles, but I think both of those are gross underestimates.
00:05:55.279 --> 00:06:07.680
Uh that everybody from gardeners to former police officers to pilots to sort of traffic control experts, it featured dozens of times in the British press, and the people don't exist.
00:06:07.759 --> 00:06:14.160
It's often AI-generated image, AI generated copy, and yeah, the person's just not there at all.
00:06:14.480 --> 00:06:17.519
I mean, I've been following this since the beginning when you started it.
00:06:17.600 --> 00:06:21.199
I think it's about, is it 18 months, two years you've been looking at this?
00:06:21.519 --> 00:06:23.759
It's just coming up to a year since we published the.
00:06:24.160 --> 00:06:28.399
And we've had, well, as you say, Barbara Santini, Rebecca Lee.
00:06:28.560 --> 00:06:29.759
She's a good one, isn't she?
00:06:29.920 --> 00:06:35.360
Because one of the interesting things about this is the way that they're sort of credentialized online.
00:06:35.759 --> 00:06:44.000
So correct me if I'm wrong, but wasn't she a an expert on the environment, or was she opining on the carbon footprint of avocados?
00:06:44.639 --> 00:06:55.439
Yeah, she the thing was that with that one, I think the people really weren't making that much effort because she offered opinions on absolutely everything and had been quoted in really quite high-profile publications.
00:06:55.519 --> 00:07:03.120
The thing was that it was on that avocado story, the reason I looked at it was it was actually quite hard to find an expert.
00:07:03.279 --> 00:07:16.800
I mean, I got one in the end, but then when I saw this one, I was like, okay, this person, they'd made mistakes with her because they had a different profile picture on some of the pieces, and uh on other ones it was that same profile picture but with a different name.
00:07:16.959 --> 00:07:34.800
And I mean, the reason people are doing this is because backlinks from sites with high reputation, which most newspapers have a sort of high domain reputation, backlinks from sites with a high domain ranking are extremely valuable.
00:07:34.959 --> 00:07:47.600
So what these people want to do is they're sending out hundreds or thousands of responses in an effort to secure links on newspaper websites, which will in turn boost the search engine rankings of sites.
00:07:47.759 --> 00:07:52.800
It's pretty trashy and it's very hard to get links in websites with good reputations.
00:07:52.959 --> 00:07:58.160
So casino sites, vape shops, sex shops, etc., etc.
00:07:58.480 --> 00:08:10.399
But as we sort of as we went into the sort of second phase of it, we realized that some of these guys are clearly selling their services to other businesses like plumbing websites, travel agents, et cetera, et cetera.
00:08:10.560 --> 00:08:23.839
And I mean, what's never been clear is whether any of these sites knew what they were doing or whether they bought services from an an SEO agency who just went, we'll improve your ranking, and then didn't tell them this is what we're gonna do.
00:08:24.240 --> 00:08:45.120
Well, that's a very interesting question because correct me if I'm wrong, but I think on the case of Jesse Chambers, who I think was a prolific travel writer, not I think you went to, if you can talk about this, to the people who were kind of running her, and they said, Oh, it's just a representative voice.
00:08:45.200 --> 00:08:49.440
Um, she represents a collective of our expertise.
00:08:49.600 --> 00:08:58.960
So was there, I mean, I don't know if you can tell our listeners a bit more to the background of that story because it seemed a little bit of a mere culpa, although also still quite murky.
00:08:59.759 --> 00:09:00.240
Totally.
00:09:00.399 --> 00:09:11.919
And I mean, I have to say, if I had been offering global work and travel PR advice, my main piece of PR advice would be ignore him, just don't say anything, it will make it worse.
00:09:12.159 --> 00:09:24.559
And uh, but they went ahead and talked to me, and they did make it worse because I mean it's just mind-bogglingly stupid argument that it's just okay to synthesize the opinions of everybody.
00:09:24.639 --> 00:09:28.240
And then they also admitted that they were using AI to write it.
00:09:28.399 --> 00:09:36.000
So they they uh they're synthesizing the opinions of all the real people in their marketing department, but also using AI.
00:09:36.159 --> 00:09:48.879
And it was just, you know, it was it was classic because it's it's just a really bad AI-generated profile picture, no personal experience, no sense of lived experience, just really painfully average web content.
00:09:48.960 --> 00:10:02.000
And then they were target targeting newspapers with just really, really trashy, wannabe viral stories like what happens when you die on a cruise ship and what happens when you fart on a plane.
00:10:02.320 --> 00:10:05.360
Obviously, you've uncovered it now, so people are much more aware of it.
00:10:05.519 --> 00:10:14.960
But it was obviously getting into not these examples specifically, but a lot of these examples were getting into the Times, the Sun, the Mirror, the tabloids, the broadsheets.
00:10:15.200 --> 00:10:18.799
What is the process whereby this is not kind of thoroughly checked?
00:10:19.200 --> 00:10:25.039
I mean, I think it's sadly a reflection of where journalism is in 2026.
00:10:25.279 --> 00:10:27.440
You know, a lot of newsrooms have been gutted.
00:10:27.519 --> 00:10:32.240
In a lot of places, there's a real culture of, you know, do 10 stories a day or you're fired.
00:10:32.559 --> 00:10:41.440
And also young journalists who are actually putting up most of the stories, you know, they're they're treated more like data entry clerks than anything else.
00:10:41.600 --> 00:10:49.200
So it's like questioning a story probably isn't gonna win you anything in the newsroom bar being shouted at by somebody.
00:10:49.360 --> 00:10:54.960
But, you know, in the a lot of these cases, the experts were just disgracefully bad.
00:10:55.120 --> 00:10:59.200
Like, you know, they it was just so blatantly obvious that they are AI generated.
00:10:59.279 --> 00:11:06.240
I mean, my my best example is Fiona Jenkins, who worked for the website Price Your Job.
00:11:06.399 --> 00:11:13.919
And she's allegedly a gardener, but there is no picture of her anywhere in a garden or even wearing gardening gloves.
00:11:14.080 --> 00:11:22.799
And the only profile image is a just staggeringly bad AI-generated headshot with just a plain grey background.
00:11:22.879 --> 00:11:25.120
I mean, it's just it's appalling.
00:11:25.279 --> 00:11:31.679
And she featured 170 times in the British press, far more than any real gardener.
00:11:31.840 --> 00:11:36.000
And she was just focusing on stories that people knew would go viral.
00:11:36.080 --> 00:11:44.960
So it's like search-based queries, like when should I stop mowing my lawn for winter, etc., etc., just to sort of capture those search terms.
00:11:45.120 --> 00:11:52.240
And the problem is that editorially, people are making the decision, this sort of content is performing well, let's just do more of it.
00:11:52.399 --> 00:11:58.480
And then it gets more and more links for this site, and you know, that gives them the incentive to do it.
00:11:58.720 --> 00:12:04.080
The guy behind this company was paying Scission, which is a press release service, to host.
00:12:04.240 --> 00:12:14.080
I mean, they normally did about two or three press releases a day, and the in total that company had around 700 features in the British press.
00:12:14.240 --> 00:12:19.600
So for Price Your Job, they were also using journalist response services like response source.
00:12:19.679 --> 00:12:29.919
So they're paying out a decent amount of money, but the amount of value that you can get in terms of search engine optimization, it's it's it's hugely valuable.
00:12:30.000 --> 00:12:35.600
So it's it's worth it for these organizations to target sort of depleted newsrooms.
00:12:35.759 --> 00:12:38.320
I mean, I I've sort of described it as almost like a cybercrime.
00:12:38.480 --> 00:12:46.639
They've found a vulnerability, which is that these sites want to run viral content and you know they are targeting that.
00:12:46.720 --> 00:12:54.320
I mean, Barbara Santini, for example, used to just answer frankly bizarre questions that had been put out by newsrooms, and she would always jump on them.
00:12:54.399 --> 00:12:58.080
It's like, what are the psychological benefits of playing dots?
00:12:58.399 --> 00:13:05.039
Now, no real psychiatrist is actually going to answer that question because it's so enormously stupid.
00:13:05.200 --> 00:13:11.440
But Barbara Santini can get a link by answering it in the persona of a sort of psychiatrist.
00:13:11.840 --> 00:13:18.399
But presumably there's a lot of harm that could be done there, particularly if you're a psychologist and you are AI generated.
00:13:18.559 --> 00:13:25.759
I mean, we know that these LLMs hallucinate, so you're literally hallucinating advice that's going straight into press media.
00:13:26.159 --> 00:13:26.559
Absolutely.
00:13:26.639 --> 00:13:28.399
And I mean, and that's widespread.
00:13:28.559 --> 00:13:37.600
There was another case where it's a sort of sex advice person, and I believe that they're using an LLM to generate that, but they're giving medical advice.
00:13:37.759 --> 00:13:45.279
And you know, when you give medical advice, I mean, particularly with Barbara Santini, she's giving advice in publications that are read by millions of people.
00:13:45.440 --> 00:13:47.759
And it's advice that is just made up.
00:13:47.919 --> 00:13:50.960
And, you know, when you do that, you can kill people.
00:13:51.200 --> 00:13:55.039
These guys are deliberately targeting stuff where it seems slightly low stakes.
00:13:55.200 --> 00:14:00.799
It's targeting kind of soft feature content in order to get those links that are so valuable to them.
00:14:01.039 --> 00:14:07.039
But, you know, the thing is that when it's medical advice, you know, it's incredibly dangerous to give that out.
00:14:07.120 --> 00:14:08.879
And a lot of the advice is just bad.
00:14:09.039 --> 00:14:13.679
You know, these guys are going for viral content at the expense of absolutely everything else.
00:14:13.759 --> 00:14:27.200
So, for instance, there was a pest control expert who also worked for Price Your John, who'd been featured more than 50 times, and he was giving advice like the best way to keep rats away from your house is to have onions, because rats can't stand the smell of onions.
00:14:27.360 --> 00:14:38.080
And I contacted the British Pest Control Association, who said that his advice was just ridiculous, old wives' tales, and no pest control professional would actually say that.
00:14:38.240 --> 00:14:48.559
I mean, you know, I'm not sure whether people really were going out and sort of putting onions around the perimeter of their homes to keep rats away, but it is it's just irresponsible, bad advice.
00:14:48.720 --> 00:15:10.159
And the problem is that the sheer volumes at which this stuff was going up, it was swamping out real advice from real people, making it harder when you go on the internet or when you, for instance, use a tool like ChatGPT, which reads available content, it means that the actual human advice is becoming more and more unavailable.
00:15:10.559 --> 00:15:18.159
I guess my favorite one of all of them, and there might have been some since that I haven't picked up on, but the royal cleaner.
00:15:18.399 --> 00:15:22.240
I'm sorry, that is I mean, that is sort of comedic and tragic.
00:15:22.320 --> 00:15:24.159
Do you want to explain what happened there?
00:15:24.480 --> 00:15:25.279
Yeah, sure.
00:15:25.440 --> 00:15:29.840
I mean, I still don't know who's behind those ones, but I have a suspicion.
00:15:30.080 --> 00:15:33.519
And they came from a fake PR agency called Signal the News.
00:15:33.679 --> 00:15:38.480
So if you go to Signal the News' website, it's kind of like half finished, it's like a template website.
00:15:38.639 --> 00:15:42.960
And people from Signal the News were sending out like dozens and dozens of fake people.
00:15:43.200 --> 00:16:01.679
They came to our attention because there was a story about lottery winners, all of whom had lost their lottery tickets in amusing and sort of quirky ways, like their child had thrown it in the bin, their dog ate it, and then lo and behold, there was some great advice from a casino website called Play Casino, and there was links to Play Casino in it.
00:16:01.840 --> 00:16:15.679
And because of that, Signal the News were on our radar, and I'd had response from quite a few journalists on tabloids and uh broadsheets saying that they were just getting submerged in emailed press releases from Signal the News.
00:16:15.840 --> 00:16:24.879
And so I was sort of chasing to do a follow-up when I noticed that one of the stories that had come from Signal the News was about a royal cleaner.
00:16:24.960 --> 00:16:34.240
And this one, I mean, it was there was a photo of a a woman, it didn't it didn't actually come up as AI generated, but to me it looks AI, so I suspect they must have masked it.
00:16:34.480 --> 00:16:36.639
Is it the only photo available of her?
00:16:36.879 --> 00:16:38.000
Yes, yeah, right.
00:16:38.320 --> 00:16:39.519
There's only one photo.
00:16:39.759 --> 00:16:42.240
I mean, that's that's a sort of hallmark of all of these.
00:16:42.399 --> 00:16:51.679
No social media, no LinkedIn, no trace of the person anywhere else, and only one photo of them, which is uh used everywhere.
00:16:51.840 --> 00:16:59.759
And I mean, in this case, she was sort of offering viral cleaning tips and little bits of insider information about the royal family.
00:16:59.919 --> 00:17:11.440
And I mean, she got uh an interview in the paper edition of the Times, which I mean that is a hard piece of coverage to get, and you know, they clearly just hadn't done due diligence on checking it.
00:17:11.519 --> 00:17:20.880
I mean, it used to, you know, I'm pretty ancient, and I remember the days when newspapers used to check every single fact, at least double source it, at least twice.
00:17:21.039 --> 00:17:29.440
I remember checking pieces on the mail, and we were told you had to double source everything, and we should count ourselves lucky because on the telegraph you had to triple source everything.
00:17:29.599 --> 00:17:31.599
And I mean, clearly that's just not being done.
00:17:31.680 --> 00:17:33.039
And she she went everywhere.
00:17:33.119 --> 00:17:40.799
There was she had things about the the cleaning products that Prince Charles bans in Buckingham Palace, Kate Middleton's morning routine.
00:17:41.039 --> 00:17:47.920
I mean, I I was baffled by that one because I mean, how a cleaner is supposed to actually be privy to Kate Middleton's morning routine.
00:17:48.000 --> 00:17:52.400
What is she hiding in the room when Kate comes in and uh does it?
00:17:52.559 --> 00:18:05.759
I mean, you know, it's like I I mean it was just baffling, but I mean she was she featured more than 20 times in newspapers dispensing sort of viral tips about Buckingham Palace and their cleaning routines, etc.
00:18:05.920 --> 00:18:06.240
etc.
00:18:06.400 --> 00:18:14.799
I mean, and that I mean this organization was putting out stuff from wellness experts, police officers, chefs, all of them didn't exist.
00:18:15.119 --> 00:18:22.160
And uh all of them were promoting clients like edicts suits, plumworld, ski vertigo, etc.
00:18:22.319 --> 00:18:22.559
etc.
00:18:22.799 --> 00:18:31.920
I mean, I knew for a fact that Signal the News or other organizations related to it are still sending press releases now because somebody sent me them last week.
00:18:32.000 --> 00:18:34.960
I mean, that's a part of the problem here, is there's no punishment.
00:18:35.279 --> 00:18:37.599
Well, that was going to be my question, actually.
00:18:37.759 --> 00:18:40.480
Who regulates and then who enforces it?
00:18:40.559 --> 00:18:44.319
Is this the domain of IPSO or someone else?
00:18:44.720 --> 00:18:56.480
IPSO have commented on it, and obviously there's now a campaign from the PRCA and the MCIPR, which are the two sort of main trade bodies for public relations in the UK.
00:18:56.720 --> 00:19:01.119
There's a campaign against this sort of deceptive practice led by them.
00:19:01.440 --> 00:19:06.400
But the problem is that it's very hard to regulate against this stuff.
00:19:06.640 --> 00:19:09.599
A lot of the stuff is coming from outside Britain.
00:19:09.839 --> 00:19:16.799
For instance, the guy who actually ran Barbara Santini is in Kenya, and the people who own that company are in Lithuania.
00:19:17.039 --> 00:19:26.400
And I was speaking to one of the journalist response services, and they were saying they dealt with between 70 and 100 fake people per day.
00:19:26.640 --> 00:19:32.480
And the the two main countries from which these things are coming are Pakistan and Bielorrussia.
00:19:32.720 --> 00:19:48.000
And so these guys are just sort of aggressive link-building SEO guys who are using this vulnerability to build sites, sometimes with a view to selling them on, or they build up sites to drive affiliate revenue.
00:19:48.160 --> 00:19:57.119
And so they have spotted a weakness, and it's very, very hard to regulate against people who are you know in countries that you know would don't extradite.
00:19:57.279 --> 00:19:58.880
I mean, in the case of Bielorrussia.
00:19:59.039 --> 00:20:01.759
And so it it's almost like cybercrime.
00:20:01.839 --> 00:20:04.640
It's extremely hard for these guys to be punished.
00:20:04.799 --> 00:20:10.799
I mean, the one thing that I hope our campaign has done is it's punished some of the brands that have used this.
00:20:11.119 --> 00:20:11.680
Like who?
00:20:11.920 --> 00:20:20.400
Well, if you search for my job quote, one of the first things that comes up is our article explaining that they put 700 fake experts into the British media.
00:20:20.559 --> 00:20:27.519
So it's going to make you quite a lot less likely to book services through My Job Quote because they're seen as a disreputable company.
00:20:27.680 --> 00:20:33.920
But I feel like there needs to be more kind of regulation and enforcement on this.
00:20:34.000 --> 00:20:41.839
Because, like I say, Signal the News, which carried out all the most outrageous ones, the fake lottery winners and the Royal Cleaner.
00:20:42.160 --> 00:20:55.519
Signal the News is still sending press releases to, well, um, one of the editors on The Sun was sending me her inbox just a couple of weeks ago, and it's still absolutely brimful of trashy press releases from Signal the News.
00:20:55.920 --> 00:21:18.480
You see, this is what I think is one of the perversions or paradoxes, if you want to be a bit kinder, about the online harms bill and this sort of government policy and regulation, is that we as real people are being asked to identify or authenticate ourselves, even just to use, you know, a social media network in future, right now and in more and more and more in the future.
00:21:18.720 --> 00:21:26.240
Meanwhile, there's this entire sort of parallel universe of complete AI identity slop.
00:21:26.400 --> 00:21:28.559
And the two are sort of coexisting.
00:21:28.640 --> 00:21:35.440
And by us being asked to identify ourselves isn't really helping eradicate any of the other stuff.
00:21:35.680 --> 00:21:38.400
They're like two completely different worlds coexisting.
00:21:38.799 --> 00:21:39.200
Totally.
00:21:39.279 --> 00:21:53.519
And well, I feel that when it comes to, you know, for instance, PR organizations who want to get their clients into publications, being able to prove that people is real is now a sort of added value situation.
00:21:53.759 --> 00:21:59.200
You know, I I know at the times they took the royal cleaner extremely seriously.
00:22:00.160 --> 00:22:09.039
Of the organizations I work for have instituted new policies, which mean that you have to authenticate every single expert that you interview before you do it.
00:22:09.200 --> 00:22:15.200
I mean, you know, hopefully that will reverse some of this culture around, oh, just get the story up, because that's been the problem.
00:22:15.279 --> 00:22:19.599
I mean, you know, there's plenty of them where they were just the expert was just anonymous.
00:22:19.759 --> 00:22:24.799
There's like stories which ran in newspapers which went, which quoted our pilot.
00:22:25.039 --> 00:22:27.680
And you're like, no, no, no, which pilot?
00:22:27.920 --> 00:22:32.160
I mean, it's just, you can't just say our pilot.
00:22:32.319 --> 00:22:35.759
And uh it uh, you know, I mean, that really shocked me.
00:22:35.920 --> 00:22:46.079
Because each of these stories, it'll have been spotted by a copy taster, it'll have been handed to somebody on the back bench on the publication, and then it'll have gone to a journalist.
00:22:46.319 --> 00:22:52.079
And all of those three people thought it was okay to publish a story with an anonymous expert.
00:22:52.160 --> 00:22:59.200
And it's just like, well, you know, what's the point of a story where the whole story is the expertise, but you don't know where it comes from?
00:22:59.519 --> 00:23:03.759
Am I right in thinking that that pilot was giving advice on the best times to travel?
00:23:04.000 --> 00:23:04.240
Yes.
00:23:04.480 --> 00:23:05.519
Or something like that.
00:23:05.680 --> 00:23:09.359
I mean, you I mean, that wouldn't happen anyway, would it?
00:23:09.519 --> 00:23:26.319
Um, if one has the time to stop and think, your point about the newsroom being, you know, working at such speed and under such pressure, actually, sometimes if you sit back and actually think about it, you would say your own brain, your own cognition would say, Well, that doesn't really scan, that doesn't really make sense.
00:23:26.400 --> 00:23:29.039
Why is a pilot, you know, commenting on that?
00:23:29.200 --> 00:23:32.559
They wouldn't be asked to comment on that and they wouldn't have a view on that necessarily.
00:23:32.880 --> 00:23:33.440
Totally.
00:23:33.599 --> 00:23:41.279
I mean, yeah, I mean, pilots probably book flights less than anybody else because you know they have to do it for their job.
00:23:41.519 --> 00:23:46.000
It's not like they're sort of going to be sitting there obsessing over when's the best time to book.
00:23:46.480 --> 00:23:48.000
I mean, it's just baffling.
00:23:48.079 --> 00:23:58.880
I mean, the ones the ones that really, really made me laugh were the ones where, so they're just desperate to get the link here, and they will just marry up an expert with something that they have nothing to do with.
00:23:59.039 --> 00:24:10.880
So one of the ones I spotted in the last lot was it was a supposedly Royal Horticultural Society trained gardener who was working for a personalized number plate website.
00:24:11.039 --> 00:24:12.640
And you're like, what?
00:24:13.359 --> 00:24:19.279
And but that was going into newspapers and getting links for these guys.
00:24:19.599 --> 00:24:23.119
I know the world's post-normal, but that really is stretching it, isn't it?
00:24:23.440 --> 00:24:24.480
It's just extraordinary.
00:24:24.720 --> 00:24:35.440
But it does sort of one can imagine a world in which these companies don't really exist, these ones that sit behind other companies, that sit behind other companies, they don't really exist.
00:24:35.680 --> 00:24:53.200
They're pumping out content created by AI-generated people, quotes that are going into media that's probably AI generated itself contextually, and then maybe even being written up by an AI journalist.
00:24:53.359 --> 00:25:01.039
So I'm like, this is it literally a fantastical non-reality that sort of exists.
00:25:01.440 --> 00:25:02.559
Yeah, no, absolutely.
00:25:02.799 --> 00:25:12.640
It is just baffling, and I I really think that the the serious part of this is that people need to put the brakes on and ensure there is human content.
00:25:12.799 --> 00:25:16.559
Otherwise, I mean, I think we're all seeing it in the internet around us.
00:25:16.640 --> 00:25:23.440
The sort of human touch is disappearing from it and AI is becoming the sort of new normal.
00:25:23.599 --> 00:25:38.160
And I think that people really, really journalists, PR people, anybody in the information ecosystem needs to fight really hard to keep humans in it and to keep human expertise at the forefront.
00:25:38.559 --> 00:25:46.640
But you recently did a story, I think, about Brian and Callum, who were AI created authors or journalists, is that correct?
00:25:46.960 --> 00:25:54.640
Yes, now this one, I mean, this uh because uh the organization involved is extremely litigious, I'll uh not say their name.
00:25:54.799 --> 00:26:15.759
But the organization behind that, they've made a habit of buying websites and then they shift to the editorial focus of the website and turn it towards casino content, reviewing casinos, and then they put links for casinos into it, which earns them money from affiliate revenue and and boosts the search engine rankings, etc.
00:26:15.920 --> 00:26:17.279
etc., for these websites.
00:26:17.519 --> 00:26:26.000
And uh in the past, they bought up lots of gambling websites, uh, they've bought lots of sports websites and they're now buying video game websites.
00:26:26.160 --> 00:26:34.480
I mean, what is truly depressing about it is a lot of these were viable businesses, but they're bought out and sacrificed just to get search engine boosts.
00:26:34.559 --> 00:26:43.680
And the amount of money in it can be calculated by that they they bought a very famous gambling website, and apparently they paid 12 million pounds for it.
00:26:43.920 --> 00:27:10.480
So, you know, they're paying market rates or over to buy up publications, then they replace all the journalists with AI and shift it to casino content in order to drive people to non-GamStop casinos, which is uh Gamstop is a voluntary thing to stop problem gamblers visiting your website, but there are non-GamStop casinos which are basically designed to target problem gamblers, and uh these are that's what these guys are advertising.
00:27:10.960 --> 00:27:18.559
So, I mean, we talked about the purposes of this and getting the bank backlinks and rising yourself up the rankings.
00:27:18.799 --> 00:27:20.160
Is Google doing enough?
00:27:20.319 --> 00:27:29.359
I mean, you do report on and obviously explain really well the way in which Google has its experience, expertise, authority, and trust sort of vectors.
00:27:29.680 --> 00:27:31.839
Is that being modified or evolved?
00:27:32.000 --> 00:27:34.720
How does Google take account of this?
00:27:34.960 --> 00:27:38.799
And are they taking enough accountability or responsibility for it, in your view?
00:27:39.200 --> 00:27:40.799
Yes, I don't think they're doing enough at all.
00:27:40.960 --> 00:27:48.160
But you know, the the day when uh Google actually gives a damn about the publishing industry is the day when hell freezes over.
00:27:48.480 --> 00:27:52.400
They have never cared about publishers and I don't think they're gonna start.
00:27:52.559 --> 00:27:53.920
I don't think they're doing enough.
00:27:54.079 --> 00:27:57.279
I think that they are the people who could really change this.
00:27:57.759 --> 00:28:07.680
And, you know, they single-handedly could get rid of this if they sort of really clamped down on this practice and clamped down on the domains using it.
00:28:07.920 --> 00:28:10.480
Not the people running it, but the people who are actually paying for it.
00:28:10.559 --> 00:28:17.920
So if you know your plumbing website pays for a dodgy SEO that uses this tactic, then they should be downranked.
00:28:18.559 --> 00:28:22.000
And uh I do think there are some manual penalties going out.
00:28:22.079 --> 00:28:28.960
So, for instance, the gambling organization that we were just talking about, a lot of their websites have been manually penalised by Google.
00:28:29.039 --> 00:28:32.640
So there is penalties out there, but I just don't think they're being used enough.
00:28:32.960 --> 00:28:33.279
Yeah.
00:28:33.519 --> 00:28:35.759
I I guess it's all about incentives, isn't it?
00:28:35.920 --> 00:28:43.119
If your job depends on not doing enough or not understanding how to fix the problem, then you'll never fix the problem.
00:28:43.359 --> 00:29:00.240
But you do talk about and work with, I think, some, would you say they're forensic digital forensics companies or sort of companies that are studios that are looking at trying to find deceptive PR, maybe these synthetic identities.
00:29:00.480 --> 00:29:02.240
How evolved is that?
00:29:02.480 --> 00:29:03.519
Is it that helpful?
00:29:03.680 --> 00:29:05.039
Is it that sophisticated?
00:29:05.200 --> 00:29:07.759
And where do you think that might go in the future?
00:29:08.160 --> 00:29:19.839
So, yeah, I I mean I work with people who are just using digital tools to track down this kind of working with another set of people who are helping me sort of hopefully zoom in on more of these.
00:29:19.920 --> 00:29:24.799
Because I I mean, I like I say, I think we've only seen the tip of the iceberg.
00:29:24.880 --> 00:29:27.920
I think there's quite a few more of these out there.
00:29:28.160 --> 00:29:30.400
So I'm looking for more of these.
00:29:30.480 --> 00:29:38.799
I mean, I think that the guys who I worked with, there are sort of uh Neo Mam studios, they they just hand a sort of bunch of data guys who were happy to work on it.
00:29:38.880 --> 00:29:41.200
And they're in the digital PR space.
00:29:41.359 --> 00:29:55.839
And I think within that industry, I mean, I've had a good response from a lot of people, within that industry, there's a lot of appetite to see this stuff stamped out because they correctly, I think, see it as quite existential for them.
00:29:56.000 --> 00:30:06.160
That if people are so annoyed with just this avalanche of just fake, rubbishy content, then they're not going to deal with any PR agencies, regardless of what they're selling.
00:30:06.319 --> 00:30:11.680
So uh, you know, there's a lot of appetite within that industry to push these bad actors out.
00:30:12.079 --> 00:30:40.319
You see, my well, there's lots of concerns and worries about this, but one of them is that it reaches up, you know, away from sort of mainstream news, which is bad enough, but into academic publishing, that these sort of fake experts really do enter at the top of academia somehow, especially when people are much more connected globally and there's many more experts, you know, over in different countries across the world, you're never necessarily going to meet them face to face, but they might have a profile.
00:30:40.480 --> 00:30:55.119
They might have a profile on a like a research site, and they might build up those contextual links to make it look like they are a real person, a real academic, and then pump out, you know, lots of academic knowledge, put in scare quotes.
00:30:55.279 --> 00:31:14.720
And the more and more of that that exists, and I think we saw a lot of it during the pandemic, you know, people coming out with expert opinions and then being sort of grabbed into the media, but they were they were hosted because they were academics, because they were seen to be much more trustworthy than perhaps anybody you would see as a sort of more mainstream commentator.
00:31:14.880 --> 00:31:20.240
Now, have you noticed any infiltration into those sorts of layers of expertise?
00:31:20.720 --> 00:31:27.359
I think that uh there's always been an element of that, because there's that temptation to become a media personality.
00:31:27.519 --> 00:31:33.599
And, you know, in every case, I always just think, okay, what is this person getting out of it?
00:31:33.839 --> 00:31:42.480
Like, you know, so when you it's an academic and they suddenly become very, very prolific in the media space, they are looking to earn money in a way that is not academia.
00:31:42.720 --> 00:31:49.039
So, you know, suddenly a guy who's a very serious gut health researcher is like producing supplements or whatever.
00:31:49.200 --> 00:31:52.240
And that there's always been that sort of permeability.
00:31:52.319 --> 00:32:03.680
I mean, I think one of the things that we're gonna see next time, around, is that we're gonna see fake experts that are just a lot better because quite a lot of the first round, they really didn't make that much effort with the fake expert.
00:32:04.000 --> 00:32:19.839
And so I think what we will you'll see is going forward is they'll make the LinkedIn page, they'll put them on the company website, they'll create fake social media for them, and it just becomes a lot less easy to see that this person isn't real.
00:32:20.240 --> 00:32:21.200
I totally agree with that.
00:32:21.279 --> 00:32:31.920
I think if you look at what 11 labs are are doing, and not to cast any aspersions on 11 labs, it's just the technology that people are using to adapt to you know whatever uses they have in mind.
00:32:32.079 --> 00:32:37.599
But it's not just now sort of synthetic voices, it's the whole synthetic world that goes around that.
00:32:37.759 --> 00:32:41.359
So they're they're going to create the tools for people to world build.
00:32:41.599 --> 00:33:06.400
It'll be like being in Roblox, I guess, you know, that you will have, you know, your friends around you, you will have a situational context that has been created by AI, generated synthetically, and it's going to be really, really difficult not just to compare and contrast and authenticate people and a very specific person against an AI version or a bot with the whole of the world that they've created.
00:33:06.640 --> 00:33:08.799
So it's going to get really, really difficult.
00:33:09.200 --> 00:33:10.880
Yes, I'm I mean I 100%.
00:33:11.200 --> 00:33:13.599
It's this is something that's going to get a lot worse.
00:33:13.839 --> 00:33:21.839
I was interviewing somebody who's a very uh who's a big expert at deepfakes, and I was like, Well, how good is deepfake technology?
00:33:21.920 --> 00:33:24.000
And she goes, Oh, well, it's this good.
00:33:24.079 --> 00:33:27.920
She pressed the button and turned into a completely different person with a different voice.
00:33:28.160 --> 00:33:38.640
And I was like, Oh my god, it literally, had I not been known that the second person wasn't real, I would have thought they were 100% real.
00:33:38.799 --> 00:33:44.319
And that technology's not exotic, it's not expensive, it's not limited, it's there.
00:33:44.559 --> 00:33:47.200
And you know, people can use it and they are using it.
00:33:47.279 --> 00:33:54.799
I think we're gonna see a world in which proving that you're real is gonna become a big asset for people.
00:33:55.119 --> 00:33:55.839
Yeah, totally.
00:33:56.000 --> 00:33:57.519
Yeah, it's been coming a long time.
00:33:57.680 --> 00:34:18.559
Now, one of the other consequences of this is in this world of machine readability, where obviously this is a whole industry, a new industry growing up to make itself machine readable and therefore visible in the online environment, mainly through SEO now, but potentially in the future through LLMs and and agentic AI?
00:34:18.960 --> 00:34:21.360
Real experts are getting crowded out.
00:34:21.679 --> 00:34:23.440
Do you have any points of view?
00:34:23.599 --> 00:34:25.199
I mean, you're you're an expert.
00:34:25.360 --> 00:34:34.480
You have worked in all the well, you've worked across all the all the main media, really, and in science and technology a lot, obviously.
00:34:34.719 --> 00:34:46.400
How do you in the future sort of compete with a lot of these synthetic AI experts, some of which will be found out, but others which there won't be, and they will pass.
00:34:46.480 --> 00:34:57.920
And actually, it's becoming very, very difficult for a human to make themselves machine readable, to show up in SEO, or actually in the future to show up in these LLMs as an expert in a particular area.
00:34:58.079 --> 00:35:02.639
You can do all the code injections you like, but actually it's it's really competitive.
00:35:03.119 --> 00:35:09.760
I mean, I think the the way that humans can human experts can stand out is simply to do things that machines can't.
00:35:10.000 --> 00:35:20.239
At the minute, I'm doing a story which I can't talk much about, but it involves a number of very prolific AI journalists who uh they are they're fake people.
00:35:20.639 --> 00:35:30.559
And basically, to catch them, I'm going to get a friend on a publication to commission one of them and then ask him to go to a flesh and blood meeting.
00:35:30.800 --> 00:35:31.280
Exactly.
00:35:31.360 --> 00:35:31.760
That's it.
00:35:32.079 --> 00:35:33.760
Because that is what a machine can't do.
00:35:33.920 --> 00:35:39.599
They can't go somewhere and have a coffee with someone because you know you're gonna notice if a massive robot appears or whatever.
00:35:39.760 --> 00:35:42.880
You know, they can't do that stuff, they can't get on the phone.
00:35:43.039 --> 00:35:51.199
I mean, in this case, I think that these fake journalists, there's the people who are operating them actually impersonate the fakes.
00:35:51.280 --> 00:35:55.920
So even if I did a phone interview, they could still get away with it.
00:35:56.159 --> 00:36:01.599
But it it's the actually going somewhere, appearing, and trying to be the person that you say you are.
00:36:01.760 --> 00:36:03.679
That is what machines still can't do.
00:36:04.159 --> 00:36:09.679
Which I guess is fine if you're in the same country, but it's not so fine if you're, you know, halfway around the world.
00:36:10.000 --> 00:36:11.039
In Bellorussia, yeah.
00:36:11.199 --> 00:36:13.039
Yeah, exactly, which way we would be.
00:36:13.360 --> 00:36:15.360
So, I mean, I just think it's fascinating.
00:36:15.519 --> 00:36:19.360
This is surely going to be just become a bigger and bigger and bigger story.
00:36:19.599 --> 00:36:21.840
Two questions for you just before we finish, Rob.
00:36:22.239 --> 00:36:35.280
What can people do now if they want some tips or tricks or heuristics really for recognizing if they might be dealing with an AI-generated identity?
00:36:35.679 --> 00:36:44.400
I mean, I think that it's just we wrote an articles about what to look for on Press Gazette, but I mean, most of the time it's just common sense.
00:36:44.559 --> 00:36:50.559
I mean, a lot of these ones that are out there at the minute, you know, it's just they've made absolutely zero effort.
00:36:50.719 --> 00:36:56.960
I mean, I would say the hallmark that always marks these guys out is that there's no way to book their services.
00:36:57.360 --> 00:37:01.679
So Fiona Jenkins, the gardener, you can't book her as a gardener.
00:37:02.400 --> 00:37:04.719
Like there's no way to contact her to do that.
00:37:04.960 --> 00:37:06.880
Whereas, you know, I'm a freelance journalist.
00:37:07.039 --> 00:37:09.519
It's very easy to find me and book me.
00:37:09.920 --> 00:37:12.000
But with these guys, there's no way to do that.
00:37:12.159 --> 00:37:15.360
And it's like it's it makes no sense because people have to make a living.
00:37:15.519 --> 00:37:22.800
And uh, if you're just a talking head that all they do is comment, there is no way to actually book your services in what you're supposedly an expert in.
00:37:23.039 --> 00:37:24.719
That's an excellent tell, isn't it?
00:37:25.519 --> 00:37:35.760
And so one of the things I'm working on is as we become more and more machinable, if you like, and everything that you have just been discussing is about this world of machinability.
00:37:35.920 --> 00:37:43.679
And actually, if you can't turn humans into machines, you just invent human-like machines that converse and communicate.
00:37:43.920 --> 00:37:48.719
We need to be even more unmachinable in very important aspects of our lives.
00:37:48.800 --> 00:37:53.679
And you've talked about some, you know, with judgment, with adaptability, lots of different areas.
00:37:53.920 --> 00:37:57.039
Where do you think we will net out 10 years from now?
00:37:57.280 --> 00:38:05.039
Will everything be much more machinable and we won't really be able to recognize reality, but we're kind of in it anyway, so we have to go with the flow.
00:38:05.199 --> 00:38:11.280
Or will there have been a backlash and a kind of reversal, if you like, of unmachinable traits?
00:38:11.679 --> 00:38:13.280
I think both things would probably happen.
00:38:13.440 --> 00:38:21.760
There'll be a move towards a sort of very sort of technology-centered, AI-driven world, but at the same time, there'll be a backlash.
00:38:21.840 --> 00:38:39.440
And I think that, you know, things that have been out of fashion, like actually picking up the phone and talking to people on the phone, will come back into fashion because people will want to authenticate, they'll want to approach things in a sort of more human, less screen-mediated way.
00:38:39.599 --> 00:38:51.199
And uh, I think that we'll see a significant kind of wider cultural backlash against AI everywhere and towards a sort of more human, more flesh and blood approach.
00:38:51.440 --> 00:38:54.159
Do you think you'll still be writing about this in 10 years, then, Rob?
00:38:54.880 --> 00:38:57.760
Well, I mean, I have to say it surprised me that uh it surprised me.
00:38:57.840 --> 00:39:00.800
I'm still writing about it a year later and it doesn't seem to be going away.
00:39:00.880 --> 00:39:02.000
So uh who knows?
00:39:02.159 --> 00:39:05.119
If I'm still around in 10 years, then yeah, probably.
00:39:06.880 --> 00:39:07.360
Brilliant.
00:39:07.599 --> 00:39:09.280
It's brilliant to hear the background to it.
00:39:09.360 --> 00:39:13.920
Um, everybody's got to follow your stories because they're absolutely fascinating.
00:39:14.079 --> 00:39:21.760
And uh I think it's fair to say that we're we're populated by probably more virtual AI personas than real people.
00:39:21.920 --> 00:39:28.000
People were worried about a population explosion in the 60s and 70s, but they they didn't realise there was going to be a virtual one.
00:39:29.199 --> 00:39:29.599
Indeed.
00:39:29.920 --> 00:39:30.400
Thanks, Rob.
00:39:30.559 --> 00:39:31.360
Brilliant to speak to you.
00:39:31.440 --> 00:39:31.840
Thanks for your time.
00:39:32.079 --> 00:39:32.400
Excellent.
00:39:32.480 --> 00:39:33.119
Thanks, Tracy.
00:39:36.960 --> 00:39:39.360
My thanks to Rob Waugh for joining me.
00:39:39.519 --> 00:39:45.119
What Rob's investigation shows is that the problem isn't misinformation, it's something far deeper.
00:39:45.360 --> 00:39:47.519
Some people call it synthetic experts.
00:39:47.679 --> 00:39:50.159
I prefer to call it identity slop.
00:39:50.400 --> 00:39:59.039
Because this is the moment when personal identity becomes just another form of highly commoditised and commercialised scalable content.
00:39:59.280 --> 00:40:02.880
Produced really only because the system needs it.
00:40:03.119 --> 00:40:09.280
And that raises larger issues for all of us living in AI-mediated information systems.
00:40:09.599 --> 00:40:13.039
Mechine Dialogues is a special series from the future of you.
00:40:13.360 --> 00:40:17.679
Find us on Spotify, Apple, and YouTube, or at Mechine.com.
00:40:17.760 --> 00:40:24.960
That's mechin e.com, or at tracyfollows.com and search for podcasts.
00:40:25.199 --> 00:40:30.880
We'd love for you to leave a rating if you like the show and do subscribe so you never miss a transmission.
00:00:03.439 --> 00:00:06.240
Jesse Chambers is a travel expert.
00:00:08.880 --> 00:00:11.599
Quoted across British newspapers.
00:00:14.080 --> 00:00:22.559
She gives advice on everything from cruise ships to long-haul flights.
00:00:22.719 --> 00:00:24.000
There's just one problem.
00:00:24.559 --> 00:00:26.879
Justice Chambers doesn't exist.
00:00:30.640 --> 00:00:32.960
Welcome to the Machine Dialogues.
00:00:33.119 --> 00:00:36.000
Conversations about AI, systems, and the self.
00:00:36.159 --> 00:00:44.640
Exploring what happens to identity in an increasingly synthetic world as we evolve into machinable and unmachinable selves.
00:00:49.600 --> 00:00:55.439
Today's guest is technology journalist Rob War, a prominent British tech journalist.
00:00:55.679 --> 00:01:00.399
Rob has spent years reporting on the evolution of the internet and emerging technologies.
00:01:00.560 --> 00:01:08.400
He's written for titles including the Daily Telegraph, Daily Mail, and Metro, and previously served as science editor at the Mail Online.
00:01:08.799 --> 00:01:15.439
He also launched the podcast networker Associated Newspapers, overseeing shows across news, politics, and culture.
00:01:16.719 --> 00:01:20.319
Recently, Rob stumbled onto something unusual.
00:01:20.560 --> 00:01:24.640
A series of media experts appearing in newspapers across Britain.
00:01:24.879 --> 00:01:28.640
Travel specialists, palace insiders, lifestyle commentators.
00:01:28.879 --> 00:01:31.680
Turned out to be synthetic biographies.
00:01:31.920 --> 00:01:38.560
These were experts manufactured to give quotes to journalists, quietly inserted into the news ecosystem.
00:01:38.879 --> 00:01:50.319
In this episode, we talk about the rise of fake experts, how synthetic authority is slipping into mainstream media, and what happens when even expertise itself becomes machine generated.
00:01:50.480 --> 00:01:58.159
Because if expertise can be generated, identity can be fabricated, and then authority can be automated.
00:01:58.560 --> 00:01:59.920
That's what we're exploring.
00:02:00.079 --> 00:02:02.640
This week on machine dialogues.
00:02:07.760 --> 00:02:08.639
Rob War.
00:02:09.039 --> 00:02:11.120
Welcome so Machine Dialogues.
00:02:11.280 --> 00:02:12.400
You are the first.
00:02:12.879 --> 00:02:21.599
And I really wanted to talk to you because obviously the subject of machine dialogues is around the machinable and the unmachinable parts of the self.
00:02:21.759 --> 00:02:28.960
And you in particular have been doing this groundbreaking work on what I would call identity slop.
00:02:30.080 --> 00:02:31.199
There's a lot of it about.
00:02:31.439 --> 00:02:31.680
Yeah.
00:02:31.840 --> 00:02:49.360
I wonder, especially for the listeners, if you could take us back to the beginning and tell us what you uncovered when you were looking at these experts that were appearing in really quite, you know, quality press outlets with their varied opinions.
00:02:50.240 --> 00:02:50.639
Sure.
00:02:50.879 --> 00:02:58.960
Well, I mean, it was a weird one because the first time it happened, it happened a year before the piece, first piece I wrote for Press Gazette.
00:02:59.039 --> 00:03:04.879
I was doing some freelance work for a marketing agency, and we got approached by somebody to do a guest blog.
00:03:05.120 --> 00:03:07.759
And I looked at the person and I thought, this is really strange.
00:03:07.919 --> 00:03:09.439
This person doesn't exist.
00:03:10.240 --> 00:03:13.360
And the other people on the site were just going, No, what do you mean?
00:03:13.520 --> 00:03:17.680
What why is the and I said, really, take it from me, this person doesn't exist.
00:03:17.840 --> 00:03:19.280
I forgot all about it.
00:03:19.439 --> 00:03:30.240
And then a year later, I was using Responsours, which is a response service for journalists, where if you need somebody to talk about something, you send a request for a specific kind of expertise.
00:03:30.400 --> 00:03:37.039
And I needed a psychiatrist to talk about the psychological impact of identity fraud.
00:03:37.199 --> 00:03:47.120
And I sent out a request and I got a response from this psychiatrist, an Oxford-educated psychiatrist, no less, who called herself Barbara Santini.
00:03:47.199 --> 00:03:50.240
And the thing was that the copy seemed a little strange.
00:03:50.400 --> 00:04:05.199
It had things that I think we're all very familiar with now as a result of ChatGPT, weird sort of capitalized subheads inserted into the copy for no reason and stuff, which often are quite hallmarks of LLM writing.
00:04:05.360 --> 00:04:07.039
And I was like, this is a bit odd.
00:04:07.120 --> 00:04:12.879
But really, the reason I sort of looked into it was because I was writing for the telegraph and she worked for a sex shop.
00:04:12.960 --> 00:04:15.520
And I thought, I can't put a sex shop in the telegraph.
00:04:15.680 --> 00:04:20.079
You know, some old Tory Kerner will have a heart attack into his cornflakes.
00:04:20.319 --> 00:04:28.000
And uh so I sort of thought, oh well, I'll just look up what her other jobs are, and I'll just list something else.
00:04:28.160 --> 00:04:32.480
And then when I looked, I realized that she wasn't real, there was just no other jobs.
00:04:32.639 --> 00:04:39.759
The only thing that she existed and did was speaking to newspapers, but she'd been hugely successful.
00:04:39.920 --> 00:04:50.639
She, in fact, she'd been in newspapers 120 times, and that included all the big sort of popular ones, you know, Mail, Express, Mirror, Sun, dozens of times.
00:04:50.800 --> 00:04:57.360
And she'd even penetrated into the BBC, The Guardian, International Press, like a New York Post.
00:04:57.519 --> 00:05:06.160
And um, when we confronted the organization behind her, we got a series of extremely convincing sounding legal threats.
00:05:06.240 --> 00:05:12.240
Because uh, I won't say his name, but the the guy who's actually behind this is uh he's a qualified solicitor.
00:05:12.399 --> 00:05:14.720
So it's like the legal threats were.
00:05:15.120 --> 00:05:16.079
Is he though?
00:05:17.199 --> 00:05:24.800
They were there were they were high there were highly convincing legal threats because you get legal threats a lot of the time, but then occasionally you get ones where you're like, okay, this is serious.
00:05:24.959 --> 00:05:34.959
And uh anyway, we published the piece, and the legal threat said that they would sue immediately if we said that this person didn't exist.
00:05:35.199 --> 00:05:39.519
We said it, and then the legal suit didn't happen.
00:05:39.680 --> 00:05:43.759
And over the coming months, I began to realize that she was far from a loan.
00:05:44.079 --> 00:05:47.120
There are a huge number of fake experts.
00:05:47.360 --> 00:05:55.199
Last time we counted it up, there was more than 50 and more than a thousand articles, but I think both of those are gross underestimates.
00:05:55.279 --> 00:06:07.680
Uh that everybody from gardeners to former police officers to pilots to sort of traffic control experts, it featured dozens of times in the British press, and the people don't exist.
00:06:07.759 --> 00:06:14.160
It's often AI-generated image, AI generated copy, and yeah, the person's just not there at all.
00:06:14.480 --> 00:06:17.519
I mean, I've been following this since the beginning when you started it.
00:06:17.600 --> 00:06:21.199
I think it's about, is it 18 months, two years you've been looking at this?
00:06:21.519 --> 00:06:23.759
It's just coming up to a year since we published the.
00:06:24.160 --> 00:06:28.399
And we've had, well, as you say, Barbara Santini, Rebecca Lee.
00:06:28.560 --> 00:06:29.759
She's a good one, isn't she?
00:06:29.920 --> 00:06:35.360
Because one of the interesting things about this is the way that they're sort of credentialized online.
00:06:35.759 --> 00:06:44.000
So correct me if I'm wrong, but wasn't she a an expert on the environment, or was she opining on the carbon footprint of avocados?
00:06:44.639 --> 00:06:55.439
Yeah, she the thing was that with that one, I think the people really weren't making that much effort because she offered opinions on absolutely everything and had been quoted in really quite high-profile publications.
00:06:55.519 --> 00:07:03.120
The thing was that it was on that avocado story, the reason I looked at it was it was actually quite hard to find an expert.
00:07:03.279 --> 00:07:16.800
I mean, I got one in the end, but then when I saw this one, I was like, okay, this person, they'd made mistakes with her because they had a different profile picture on some of the pieces, and uh on other ones it was that same profile picture but with a different name.
00:07:16.959 --> 00:07:34.800
And I mean, the reason people are doing this is because backlinks from sites with high reputation, which most newspapers have a sort of high domain reputation, backlinks from sites with a high domain ranking are extremely valuable.
00:07:34.959 --> 00:07:47.600
So what these people want to do is they're sending out hundreds or thousands of responses in an effort to secure links on newspaper websites, which will in turn boost the search engine rankings of sites.
00:07:47.759 --> 00:07:52.800
It's pretty trashy and it's very hard to get links in websites with good reputations.
00:07:52.959 --> 00:07:58.160
So casino sites, vape shops, sex shops, etc., etc.
00:07:58.480 --> 00:08:10.399
But as we sort of as we went into the sort of second phase of it, we realized that some of these guys are clearly selling their services to other businesses like plumbing websites, travel agents, et cetera, et cetera.
00:08:10.560 --> 00:08:23.839
And I mean, what's never been clear is whether any of these sites knew what they were doing or whether they bought services from an an SEO agency who just went, we'll improve your ranking, and then didn't tell them this is what we're gonna do.
00:08:24.240 --> 00:08:45.120
Well, that's a very interesting question because correct me if I'm wrong, but I think on the case of Jesse Chambers, who I think was a prolific travel writer, not I think you went to, if you can talk about this, to the people who were kind of running her, and they said, Oh, it's just a representative voice.
00:08:45.200 --> 00:08:49.440
Um, she represents a collective of our expertise.
00:08:49.600 --> 00:08:58.960
So was there, I mean, I don't know if you can tell our listeners a bit more to the background of that story because it seemed a little bit of a mere culpa, although also still quite murky.
00:08:59.759 --> 00:09:00.240
Totally.
00:09:00.399 --> 00:09:11.919
And I mean, I have to say, if I had been offering global work and travel PR advice, my main piece of PR advice would be ignore him, just don't say anything, it will make it worse.
00:09:12.159 --> 00:09:24.559
And uh, but they went ahead and talked to me, and they did make it worse because I mean it's just mind-bogglingly stupid argument that it's just okay to synthesize the opinions of everybody.
00:09:24.639 --> 00:09:28.240
And then they also admitted that they were using AI to write it.
00:09:28.399 --> 00:09:36.000
So they they uh they're synthesizing the opinions of all the real people in their marketing department, but also using AI.
00:09:36.159 --> 00:09:48.879
And it was just, you know, it was it was classic because it's it's just a really bad AI-generated profile picture, no personal experience, no sense of lived experience, just really painfully average web content.
00:09:48.960 --> 00:10:02.000
And then they were target targeting newspapers with just really, really trashy, wannabe viral stories like what happens when you die on a cruise ship and what happens when you fart on a plane.
00:10:02.320 --> 00:10:05.360
Obviously, you've uncovered it now, so people are much more aware of it.
00:10:05.519 --> 00:10:14.960
But it was obviously getting into not these examples specifically, but a lot of these examples were getting into the Times, the Sun, the Mirror, the tabloids, the broadsheets.
00:10:15.200 --> 00:10:18.799
What is the process whereby this is not kind of thoroughly checked?
00:10:19.200 --> 00:10:25.039
I mean, I think it's sadly a reflection of where journalism is in 2026.
00:10:25.279 --> 00:10:27.440
You know, a lot of newsrooms have been gutted.
00:10:27.519 --> 00:10:32.240
In a lot of places, there's a real culture of, you know, do 10 stories a day or you're fired.
00:10:32.559 --> 00:10:41.440
And also young journalists who are actually putting up most of the stories, you know, they're they're treated more like data entry clerks than anything else.
00:10:41.600 --> 00:10:49.200
So it's like questioning a story probably isn't gonna win you anything in the newsroom bar being shouted at by somebody.
00:10:49.360 --> 00:10:54.960
But, you know, in the a lot of these cases, the experts were just disgracefully bad.
00:10:55.120 --> 00:10:59.200
Like, you know, they it was just so blatantly obvious that they are AI generated.
00:10:59.279 --> 00:11:06.240
I mean, my my best example is Fiona Jenkins, who worked for the website Price Your Job.
00:11:06.399 --> 00:11:13.919
And she's allegedly a gardener, but there is no picture of her anywhere in a garden or even wearing gardening gloves.
00:11:14.080 --> 00:11:22.799
And the only profile image is a just staggeringly bad AI-generated headshot with just a plain grey background.
00:11:22.879 --> 00:11:25.120
I mean, it's just it's appalling.
00:11:25.279 --> 00:11:31.679
And she featured 170 times in the British press, far more than any real gardener.
00:11:31.840 --> 00:11:36.000
And she was just focusing on stories that people knew would go viral.
00:11:36.080 --> 00:11:44.960
So it's like search-based queries, like when should I stop mowing my lawn for winter, etc., etc., just to sort of capture those search terms.
00:11:45.120 --> 00:11:52.240
And the problem is that editorially, people are making the decision, this sort of content is performing well, let's just do more of it.
00:11:52.399 --> 00:11:58.480
And then it gets more and more links for this site, and you know, that gives them the incentive to do it.
00:11:58.720 --> 00:12:04.080
The guy behind this company was paying Scission, which is a press release service, to host.
00:12:04.240 --> 00:12:14.080
I mean, they normally did about two or three press releases a day, and the in total that company had around 700 features in the British press.
00:12:14.240 --> 00:12:19.600
So for Price Your Job, they were also using journalist response services like response source.
00:12:19.679 --> 00:12:29.919
So they're paying out a decent amount of money, but the amount of value that you can get in terms of search engine optimization, it's it's it's hugely valuable.
00:12:30.000 --> 00:12:35.600
So it's it's worth it for these organizations to target sort of depleted newsrooms.
00:12:35.759 --> 00:12:38.320
I mean, I I've sort of described it as almost like a cybercrime.
00:12:38.480 --> 00:12:46.639
They've found a vulnerability, which is that these sites want to run viral content and you know they are targeting that.
00:12:46.720 --> 00:12:54.320
I mean, Barbara Santini, for example, used to just answer frankly bizarre questions that had been put out by newsrooms, and she would always jump on them.
00:12:54.399 --> 00:12:58.080
It's like, what are the psychological benefits of playing dots?
00:12:58.399 --> 00:13:05.039
Now, no real psychiatrist is actually going to answer that question because it's so enormously stupid.
00:13:05.200 --> 00:13:11.440
But Barbara Santini can get a link by answering it in the persona of a sort of psychiatrist.
00:13:11.840 --> 00:13:18.399
But presumably there's a lot of harm that could be done there, particularly if you're a psychologist and you are AI generated.
00:13:18.559 --> 00:13:25.759
I mean, we know that these LLMs hallucinate, so you're literally hallucinating advice that's going straight into press media.
00:13:26.159 --> 00:13:26.559
Absolutely.
00:13:26.639 --> 00:13:28.399
And I mean, and that's widespread.
00:13:28.559 --> 00:13:37.600
There was another case where it's a sort of sex advice person, and I believe that they're using an LLM to generate that, but they're giving medical advice.
00:13:37.759 --> 00:13:45.279
And you know, when you give medical advice, I mean, particularly with Barbara Santini, she's giving advice in publications that are read by millions of people.
00:13:45.440 --> 00:13:47.759
And it's advice that is just made up.
00:13:47.919 --> 00:13:50.960
And, you know, when you do that, you can kill people.
00:13:51.200 --> 00:13:55.039
These guys are deliberately targeting stuff where it seems slightly low stakes.
00:13:55.200 --> 00:14:00.799
It's targeting kind of soft feature content in order to get those links that are so valuable to them.
00:14:01.039 --> 00:14:07.039
But, you know, the thing is that when it's medical advice, you know, it's incredibly dangerous to give that out.
00:14:07.120 --> 00:14:08.879
And a lot of the advice is just bad.
00:14:09.039 --> 00:14:13.679
You know, these guys are going for viral content at the expense of absolutely everything else.
00:14:13.759 --> 00:14:27.200
So, for instance, there was a pest control expert who also worked for Price Your John, who'd been featured more than 50 times, and he was giving advice like the best way to keep rats away from your house is to have onions, because rats can't stand the smell of onions.
00:14:27.360 --> 00:14:38.080
And I contacted the British Pest Control Association, who said that his advice was just ridiculous, old wives' tales, and no pest control professional would actually say that.
00:14:38.240 --> 00:14:48.559
I mean, you know, I'm not sure whether people really were going out and sort of putting onions around the perimeter of their homes to keep rats away, but it is it's just irresponsible, bad advice.
00:14:48.720 --> 00:15:10.159
And the problem is that the sheer volumes at which this stuff was going up, it was swamping out real advice from real people, making it harder when you go on the internet or when you, for instance, use a tool like ChatGPT, which reads available content, it means that the actual human advice is becoming more and more unavailable.
00:15:10.559 --> 00:15:18.159
I guess my favorite one of all of them, and there might have been some since that I haven't picked up on, but the royal cleaner.
00:15:18.399 --> 00:15:22.240
I'm sorry, that is I mean, that is sort of comedic and tragic.
00:15:22.320 --> 00:15:24.159
Do you want to explain what happened there?
00:15:24.480 --> 00:15:25.279
Yeah, sure.
00:15:25.440 --> 00:15:29.840
I mean, I still don't know who's behind those ones, but I have a suspicion.
00:15:30.080 --> 00:15:33.519
And they came from a fake PR agency called Signal the News.
00:15:33.679 --> 00:15:38.480
So if you go to Signal the News' website, it's kind of like half finished, it's like a template website.
00:15:38.639 --> 00:15:42.960
And people from Signal the News were sending out like dozens and dozens of fake people.
00:15:43.200 --> 00:16:01.679
They came to our attention because there was a story about lottery winners, all of whom had lost their lottery tickets in amusing and sort of quirky ways, like their child had thrown it in the bin, their dog ate it, and then lo and behold, there was some great advice from a casino website called Play Casino, and there was links to Play Casino in it.
00:16:01.840 --> 00:16:15.679
And because of that, Signal the News were on our radar, and I'd had response from quite a few journalists on tabloids and uh broadsheets saying that they were just getting submerged in emailed press releases from Signal the News.
00:16:15.840 --> 00:16:24.879
And so I was sort of chasing to do a follow-up when I noticed that one of the stories that had come from Signal the News was about a royal cleaner.
00:16:24.960 --> 00:16:34.240
And this one, I mean, it was there was a photo of a a woman, it didn't it didn't actually come up as AI generated, but to me it looks AI, so I suspect they must have masked it.
00:16:34.480 --> 00:16:36.639
Is it the only photo available of her?
00:16:36.879 --> 00:16:38.000
Yes, yeah, right.
00:16:38.320 --> 00:16:39.519
There's only one photo.
00:16:39.759 --> 00:16:42.240
I mean, that's that's a sort of hallmark of all of these.
00:16:42.399 --> 00:16:51.679
No social media, no LinkedIn, no trace of the person anywhere else, and only one photo of them, which is uh used everywhere.
00:16:51.840 --> 00:16:59.759
And I mean, in this case, she was sort of offering viral cleaning tips and little bits of insider information about the royal family.
00:16:59.919 --> 00:17:11.440
And I mean, she got uh an interview in the paper edition of the Times, which I mean that is a hard piece of coverage to get, and you know, they clearly just hadn't done due diligence on checking it.
00:17:11.519 --> 00:17:20.880
I mean, it used to, you know, I'm pretty ancient, and I remember the days when newspapers used to check every single fact, at least double source it, at least twice.
00:17:21.039 --> 00:17:29.440
I remember checking pieces on the mail, and we were told you had to double source everything, and we should count ourselves lucky because on the telegraph you had to triple source everything.
00:17:29.599 --> 00:17:31.599
And I mean, clearly that's just not being done.
00:17:31.680 --> 00:17:33.039
And she she went everywhere.
00:17:33.119 --> 00:17:40.799
There was she had things about the the cleaning products that Prince Charles bans in Buckingham Palace, Kate Middleton's morning routine.
00:17:41.039 --> 00:17:47.920
I mean, I I was baffled by that one because I mean, how a cleaner is supposed to actually be privy to Kate Middleton's morning routine.
00:17:48.000 --> 00:17:52.400
What is she hiding in the room when Kate comes in and uh does it?
00:17:52.559 --> 00:18:05.759
I mean, you know, it's like I I mean it was just baffling, but I mean she was she featured more than 20 times in newspapers dispensing sort of viral tips about Buckingham Palace and their cleaning routines, etc.
00:18:05.920 --> 00:18:06.240
etc.
00:18:06.400 --> 00:18:14.799
I mean, and that I mean this organization was putting out stuff from wellness experts, police officers, chefs, all of them didn't exist.
00:18:15.119 --> 00:18:22.160
And uh all of them were promoting clients like edicts suits, plumworld, ski vertigo, etc.
00:18:22.319 --> 00:18:22.559
etc.
00:18:22.799 --> 00:18:31.920
I mean, I knew for a fact that Signal the News or other organizations related to it are still sending press releases now because somebody sent me them last week.
00:18:32.000 --> 00:18:34.960
I mean, that's a part of the problem here, is there's no punishment.
00:18:35.279 --> 00:18:37.599
Well, that was going to be my question, actually.
00:18:37.759 --> 00:18:40.480
Who regulates and then who enforces it?
00:18:40.559 --> 00:18:44.319
Is this the domain of IPSO or someone else?
00:18:44.720 --> 00:18:56.480
IPSO have commented on it, and obviously there's now a campaign from the PRCA and the MCIPR, which are the two sort of main trade bodies for public relations in the UK.
00:18:56.720 --> 00:19:01.119
There's a campaign against this sort of deceptive practice led by them.
00:19:01.440 --> 00:19:06.400
But the problem is that it's very hard to regulate against this stuff.
00:19:06.640 --> 00:19:09.599
A lot of the stuff is coming from outside Britain.
00:19:09.839 --> 00:19:16.799
For instance, the guy who actually ran Barbara Santini is in Kenya, and the people who own that company are in Lithuania.
00:19:17.039 --> 00:19:26.400
And I was speaking to one of the journalist response services, and they were saying they dealt with between 70 and 100 fake people per day.
00:19:26.640 --> 00:19:32.480
And the the two main countries from which these things are coming are Pakistan and Bielorrussia.
00:19:32.720 --> 00:19:48.000
And so these guys are just sort of aggressive link-building SEO guys who are using this vulnerability to build sites, sometimes with a view to selling them on, or they build up sites to drive affiliate revenue.
00:19:48.160 --> 00:19:57.119
And so they have spotted a weakness, and it's very, very hard to regulate against people who are you know in countries that you know would don't extradite.
00:19:57.279 --> 00:19:58.880
I mean, in the case of Bielorrussia.
00:19:59.039 --> 00:20:01.759
And so it it's almost like cybercrime.
00:20:01.839 --> 00:20:04.640
It's extremely hard for these guys to be punished.
00:20:04.799 --> 00:20:10.799
I mean, the one thing that I hope our campaign has done is it's punished some of the brands that have used this.
00:20:11.119 --> 00:20:11.680
Like who?
00:20:11.920 --> 00:20:20.400
Well, if you search for my job quote, one of the first things that comes up is our article explaining that they put 700 fake experts into the British media.
00:20:20.559 --> 00:20:27.519
So it's going to make you quite a lot less likely to book services through My Job Quote because they're seen as a disreputable company.
00:20:27.680 --> 00:20:33.920
But I feel like there needs to be more kind of regulation and enforcement on this.
00:20:34.000 --> 00:20:41.839
Because, like I say, Signal the News, which carried out all the most outrageous ones, the fake lottery winners and the Royal Cleaner.
00:20:42.160 --> 00:20:55.519
Signal the News is still sending press releases to, well, um, one of the editors on The Sun was sending me her inbox just a couple of weeks ago, and it's still absolutely brimful of trashy press releases from Signal the News.
00:20:55.920 --> 00:21:18.480
You see, this is what I think is one of the perversions or paradoxes, if you want to be a bit kinder, about the online harms bill and this sort of government policy and regulation, is that we as real people are being asked to identify or authenticate ourselves, even just to use, you know, a social media network in future, right now and in more and more and more in the future.
00:21:18.720 --> 00:21:26.240
Meanwhile, there's this entire sort of parallel universe of complete AI identity slop.
00:21:26.400 --> 00:21:28.559
And the two are sort of coexisting.
00:21:28.640 --> 00:21:35.440
And by us being asked to identify ourselves isn't really helping eradicate any of the other stuff.
00:21:35.680 --> 00:21:38.400
They're like two completely different worlds coexisting.
00:21:38.799 --> 00:21:39.200
Totally.
00:21:39.279 --> 00:21:53.519
And well, I feel that when it comes to, you know, for instance, PR organizations who want to get their clients into publications, being able to prove that people is real is now a sort of added value situation.
00:21:53.759 --> 00:21:59.200
You know, I I know at the times they took the royal cleaner extremely seriously.
00:22:00.160 --> 00:22:09.039
Of the organizations I work for have instituted new policies, which mean that you have to authenticate every single expert that you interview before you do it.
00:22:09.200 --> 00:22:15.200
I mean, you know, hopefully that will reverse some of this culture around, oh, just get the story up, because that's been the problem.
00:22:15.279 --> 00:22:19.599
I mean, you know, there's plenty of them where they were just the expert was just anonymous.
00:22:19.759 --> 00:22:24.799
There's like stories which ran in newspapers which went, which quoted our pilot.
00:22:25.039 --> 00:22:27.680
And you're like, no, no, no, which pilot?
00:22:27.920 --> 00:22:32.160
I mean, it's just, you can't just say our pilot.
00:22:32.319 --> 00:22:35.759
And uh it uh, you know, I mean, that really shocked me.
00:22:35.920 --> 00:22:46.079
Because each of these stories, it'll have been spotted by a copy taster, it'll have been handed to somebody on the back bench on the publication, and then it'll have gone to a journalist.
00:22:46.319 --> 00:22:52.079
And all of those three people thought it was okay to publish a story with an anonymous expert.
00:22:52.160 --> 00:22:59.200
And it's just like, well, you know, what's the point of a story where the whole story is the expertise, but you don't know where it comes from?
00:22:59.519 --> 00:23:03.759
Am I right in thinking that that pilot was giving advice on the best times to travel?
00:23:04.000 --> 00:23:04.240
Yes.
00:23:04.480 --> 00:23:05.519
Or something like that.
00:23:05.680 --> 00:23:09.359
I mean, you I mean, that wouldn't happen anyway, would it?
00:23:09.519 --> 00:23:26.319
Um, if one has the time to stop and think, your point about the newsroom being, you know, working at such speed and under such pressure, actually, sometimes if you sit back and actually think about it, you would say your own brain, your own cognition would say, Well, that doesn't really scan, that doesn't really make sense.
00:23:26.400 --> 00:23:29.039
Why is a pilot, you know, commenting on that?
00:23:29.200 --> 00:23:32.559
They wouldn't be asked to comment on that and they wouldn't have a view on that necessarily.
00:23:32.880 --> 00:23:33.440
Totally.
00:23:33.599 --> 00:23:41.279
I mean, yeah, I mean, pilots probably book flights less than anybody else because you know they have to do it for their job.
00:23:41.519 --> 00:23:46.000
It's not like they're sort of going to be sitting there obsessing over when's the best time to book.
00:23:46.480 --> 00:23:48.000
I mean, it's just baffling.
00:23:48.079 --> 00:23:58.880
I mean, the ones the ones that really, really made me laugh were the ones where, so they're just desperate to get the link here, and they will just marry up an expert with something that they have nothing to do with.
00:23:59.039 --> 00:24:10.880
So one of the ones I spotted in the last lot was it was a supposedly Royal Horticultural Society trained gardener who was working for a personalized number plate website.
00:24:11.039 --> 00:24:12.640
And you're like, what?
00:24:13.359 --> 00:24:19.279
And but that was going into newspapers and getting links for these guys.
00:24:19.599 --> 00:24:23.119
I know the world's post-normal, but that really is stretching it, isn't it?
00:24:23.440 --> 00:24:24.480
It's just extraordinary.
00:24:24.720 --> 00:24:35.440
But it does sort of one can imagine a world in which these companies don't really exist, these ones that sit behind other companies, that sit behind other companies, they don't really exist.
00:24:35.680 --> 00:24:53.200
They're pumping out content created by AI-generated people, quotes that are going into media that's probably AI generated itself contextually, and then maybe even being written up by an AI journalist.
00:24:53.359 --> 00:25:01.039
So I'm like, this is it literally a fantastical non-reality that sort of exists.
00:25:01.440 --> 00:25:02.559
Yeah, no, absolutely.
00:25:02.799 --> 00:25:12.640
It is just baffling, and I I really think that the the serious part of this is that people need to put the brakes on and ensure there is human content.
00:25:12.799 --> 00:25:16.559
Otherwise, I mean, I think we're all seeing it in the internet around us.
00:25:16.640 --> 00:25:23.440
The sort of human touch is disappearing from it and AI is becoming the sort of new normal.
00:25:23.599 --> 00:25:38.160
And I think that people really, really journalists, PR people, anybody in the information ecosystem needs to fight really hard to keep humans in it and to keep human expertise at the forefront.
00:25:38.559 --> 00:25:46.640
But you recently did a story, I think, about Brian and Callum, who were AI created authors or journalists, is that correct?
00:25:46.960 --> 00:25:54.640
Yes, now this one, I mean, this uh because uh the organization involved is extremely litigious, I'll uh not say their name.
00:25:54.799 --> 00:26:15.759
But the organization behind that, they've made a habit of buying websites and then they shift to the editorial focus of the website and turn it towards casino content, reviewing casinos, and then they put links for casinos into it, which earns them money from affiliate revenue and and boosts the search engine rankings, etc.
00:26:15.920 --> 00:26:17.279
etc., for these websites.
00:26:17.519 --> 00:26:26.000
And uh in the past, they bought up lots of gambling websites, uh, they've bought lots of sports websites and they're now buying video game websites.
00:26:26.160 --> 00:26:34.480
I mean, what is truly depressing about it is a lot of these were viable businesses, but they're bought out and sacrificed just to get search engine boosts.
00:26:34.559 --> 00:26:43.680
And the amount of money in it can be calculated by that they they bought a very famous gambling website, and apparently they paid 12 million pounds for it.
00:26:43.920 --> 00:27:10.480
So, you know, they're paying market rates or over to buy up publications, then they replace all the journalists with AI and shift it to casino content in order to drive people to non-GamStop casinos, which is uh Gamstop is a voluntary thing to stop problem gamblers visiting your website, but there are non-GamStop casinos which are basically designed to target problem gamblers, and uh these are that's what these guys are advertising.
00:27:10.960 --> 00:27:18.559
So, I mean, we talked about the purposes of this and getting the bank backlinks and rising yourself up the rankings.
00:27:18.799 --> 00:27:20.160
Is Google doing enough?
00:27:20.319 --> 00:27:29.359
I mean, you do report on and obviously explain really well the way in which Google has its experience, expertise, authority, and trust sort of vectors.
00:27:29.680 --> 00:27:31.839
Is that being modified or evolved?
00:27:32.000 --> 00:27:34.720
How does Google take account of this?
00:27:34.960 --> 00:27:38.799
And are they taking enough accountability or responsibility for it, in your view?
00:27:39.200 --> 00:27:40.799
Yes, I don't think they're doing enough at all.
00:27:40.960 --> 00:27:48.160
But you know, the the day when uh Google actually gives a damn about the publishing industry is the day when hell freezes over.
00:27:48.480 --> 00:27:52.400
They have never cared about publishers and I don't think they're gonna start.
00:27:52.559 --> 00:27:53.920
I don't think they're doing enough.
00:27:54.079 --> 00:27:57.279
I think that they are the people who could really change this.
00:27:57.759 --> 00:28:07.680
And, you know, they single-handedly could get rid of this if they sort of really clamped down on this practice and clamped down on the domains using it.
00:28:07.920 --> 00:28:10.480
Not the people running it, but the people who are actually paying for it.
00:28:10.559 --> 00:28:17.920
So if you know your plumbing website pays for a dodgy SEO that uses this tactic, then they should be downranked.
00:28:18.559 --> 00:28:22.000
And uh I do think there are some manual penalties going out.
00:28:22.079 --> 00:28:28.960
So, for instance, the gambling organization that we were just talking about, a lot of their websites have been manually penalised by Google.
00:28:29.039 --> 00:28:32.640
So there is penalties out there, but I just don't think they're being used enough.
00:28:32.960 --> 00:28:33.279
Yeah.
00:28:33.519 --> 00:28:35.759
I I guess it's all about incentives, isn't it?
00:28:35.920 --> 00:28:43.119
If your job depends on not doing enough or not understanding how to fix the problem, then you'll never fix the problem.
00:28:43.359 --> 00:29:00.240
But you do talk about and work with, I think, some, would you say they're forensic digital forensics companies or sort of companies that are studios that are looking at trying to find deceptive PR, maybe these synthetic identities.
00:29:00.480 --> 00:29:02.240
How evolved is that?
00:29:02.480 --> 00:29:03.519
Is it that helpful?
00:29:03.680 --> 00:29:05.039
Is it that sophisticated?
00:29:05.200 --> 00:29:07.759
And where do you think that might go in the future?
00:29:08.160 --> 00:29:19.839
So, yeah, I I mean I work with people who are just using digital tools to track down this kind of working with another set of people who are helping me sort of hopefully zoom in on more of these.
00:29:19.920 --> 00:29:24.799
Because I I mean, I like I say, I think we've only seen the tip of the iceberg.
00:29:24.880 --> 00:29:27.920
I think there's quite a few more of these out there.
00:29:28.160 --> 00:29:30.400
So I'm looking for more of these.
00:29:30.480 --> 00:29:38.799
I mean, I think that the guys who I worked with, there are sort of uh Neo Mam studios, they they just hand a sort of bunch of data guys who were happy to work on it.
00:29:38.880 --> 00:29:41.200
And they're in the digital PR space.
00:29:41.359 --> 00:29:55.839
And I think within that industry, I mean, I've had a good response from a lot of people, within that industry, there's a lot of appetite to see this stuff stamped out because they correctly, I think, see it as quite existential for them.
00:29:56.000 --> 00:30:06.160
That if people are so annoyed with just this avalanche of just fake, rubbishy content, then they're not going to deal with any PR agencies, regardless of what they're selling.
00:30:06.319 --> 00:30:11.680
So uh, you know, there's a lot of appetite within that industry to push these bad actors out.
00:30:12.079 --> 00:30:40.319
You see, my well, there's lots of concerns and worries about this, but one of them is that it reaches up, you know, away from sort of mainstream news, which is bad enough, but into academic publishing, that these sort of fake experts really do enter at the top of academia somehow, especially when people are much more connected globally and there's many more experts, you know, over in different countries across the world, you're never necessarily going to meet them face to face, but they might have a profile.
00:30:40.480 --> 00:30:55.119
They might have a profile on a like a research site, and they might build up those contextual links to make it look like they are a real person, a real academic, and then pump out, you know, lots of academic knowledge, put in scare quotes.
00:30:55.279 --> 00:31:14.720
And the more and more of that that exists, and I think we saw a lot of it during the pandemic, you know, people coming out with expert opinions and then being sort of grabbed into the media, but they were they were hosted because they were academics, because they were seen to be much more trustworthy than perhaps anybody you would see as a sort of more mainstream commentator.
00:31:14.880 --> 00:31:20.240
Now, have you noticed any infiltration into those sorts of layers of expertise?
00:31:20.720 --> 00:31:27.359
I think that uh there's always been an element of that, because there's that temptation to become a media personality.
00:31:27.519 --> 00:31:33.599
And, you know, in every case, I always just think, okay, what is this person getting out of it?
00:31:33.839 --> 00:31:42.480
Like, you know, so when you it's an academic and they suddenly become very, very prolific in the media space, they are looking to earn money in a way that is not academia.
00:31:42.720 --> 00:31:49.039
So, you know, suddenly a guy who's a very serious gut health researcher is like producing supplements or whatever.
00:31:49.200 --> 00:31:52.240
And that there's always been that sort of permeability.
00:31:52.319 --> 00:32:03.680
I mean, I think one of the things that we're gonna see next time, around, is that we're gonna see fake experts that are just a lot better because quite a lot of the first round, they really didn't make that much effort with the fake expert.
00:32:04.000 --> 00:32:19.839
And so I think what we will you'll see is going forward is they'll make the LinkedIn page, they'll put them on the company website, they'll create fake social media for them, and it just becomes a lot less easy to see that this person isn't real.
00:32:20.240 --> 00:32:21.200
I totally agree with that.
00:32:21.279 --> 00:32:31.920
I think if you look at what 11 labs are are doing, and not to cast any aspersions on 11 labs, it's just the technology that people are using to adapt to you know whatever uses they have in mind.
00:32:32.079 --> 00:32:37.599
But it's not just now sort of synthetic voices, it's the whole synthetic world that goes around that.
00:32:37.759 --> 00:32:41.359
So they're they're going to create the tools for people to world build.
00:32:41.599 --> 00:33:06.400
It'll be like being in Roblox, I guess, you know, that you will have, you know, your friends around you, you will have a situational context that has been created by AI, generated synthetically, and it's going to be really, really difficult not just to compare and contrast and authenticate people and a very specific person against an AI version or a bot with the whole of the world that they've created.
00:33:06.640 --> 00:33:08.799
So it's going to get really, really difficult.
00:33:09.200 --> 00:33:10.880
Yes, I'm I mean I 100%.
00:33:11.200 --> 00:33:13.599
It's this is something that's going to get a lot worse.
00:33:13.839 --> 00:33:21.839
I was interviewing somebody who's a very uh who's a big expert at deepfakes, and I was like, Well, how good is deepfake technology?
00:33:21.920 --> 00:33:24.000
And she goes, Oh, well, it's this good.
00:33:24.079 --> 00:33:27.920
She pressed the button and turned into a completely different person with a different voice.
00:33:28.160 --> 00:33:38.640
And I was like, Oh my god, it literally, had I not been known that the second person wasn't real, I would have thought they were 100% real.
00:33:38.799 --> 00:33:44.319
And that technology's not exotic, it's not expensive, it's not limited, it's there.
00:33:44.559 --> 00:33:47.200
And you know, people can use it and they are using it.
00:33:47.279 --> 00:33:54.799
I think we're gonna see a world in which proving that you're real is gonna become a big asset for people.
00:33:55.119 --> 00:33:55.839
Yeah, totally.
00:33:56.000 --> 00:33:57.519
Yeah, it's been coming a long time.
00:33:57.680 --> 00:34:18.559
Now, one of the other consequences of this is in this world of machine readability, where obviously this is a whole industry, a new industry growing up to make itself machine readable and therefore visible in the online environment, mainly through SEO now, but potentially in the future through LLMs and and agentic AI?
00:34:18.960 --> 00:34:21.360
Real experts are getting crowded out.
00:34:21.679 --> 00:34:23.440
Do you have any points of view?
00:34:23.599 --> 00:34:25.199
I mean, you're you're an expert.
00:34:25.360 --> 00:34:34.480
You have worked in all the well, you've worked across all the all the main media, really, and in science and technology a lot, obviously.
00:34:34.719 --> 00:34:46.400
How do you in the future sort of compete with a lot of these synthetic AI experts, some of which will be found out, but others which there won't be, and they will pass.
00:34:46.480 --> 00:34:57.920
And actually, it's becoming very, very difficult for a human to make themselves machine readable, to show up in SEO, or actually in the future to show up in these LLMs as an expert in a particular area.
00:34:58.079 --> 00:35:02.639
You can do all the code injections you like, but actually it's it's really competitive.
00:35:03.119 --> 00:35:09.760
I mean, I think the the way that humans can human experts can stand out is simply to do things that machines can't.
00:35:10.000 --> 00:35:20.239
At the minute, I'm doing a story which I can't talk much about, but it involves a number of very prolific AI journalists who uh they are they're fake people.
00:35:20.639 --> 00:35:30.559
And basically, to catch them, I'm going to get a friend on a publication to commission one of them and then ask him to go to a flesh and blood meeting.
00:35:30.800 --> 00:35:31.280
Exactly.
00:35:31.360 --> 00:35:31.760
That's it.
00:35:32.079 --> 00:35:33.760
Because that is what a machine can't do.
00:35:33.920 --> 00:35:39.599
They can't go somewhere and have a coffee with someone because you know you're gonna notice if a massive robot appears or whatever.
00:35:39.760 --> 00:35:42.880
You know, they can't do that stuff, they can't get on the phone.
00:35:43.039 --> 00:35:51.199
I mean, in this case, I think that these fake journalists, there's the people who are operating them actually impersonate the fakes.
00:35:51.280 --> 00:35:55.920
So even if I did a phone interview, they could still get away with it.
00:35:56.159 --> 00:36:01.599
But it it's the actually going somewhere, appearing, and trying to be the person that you say you are.
00:36:01.760 --> 00:36:03.679
That is what machines still can't do.
00:36:04.159 --> 00:36:09.679
Which I guess is fine if you're in the same country, but it's not so fine if you're, you know, halfway around the world.
00:36:10.000 --> 00:36:11.039
In Bellorussia, yeah.
00:36:11.199 --> 00:36:13.039
Yeah, exactly, which way we would be.
00:36:13.360 --> 00:36:15.360
So, I mean, I just think it's fascinating.
00:36:15.519 --> 00:36:19.360
This is surely going to be just become a bigger and bigger and bigger story.
00:36:19.599 --> 00:36:21.840
Two questions for you just before we finish, Rob.
00:36:22.239 --> 00:36:35.280
What can people do now if they want some tips or tricks or heuristics really for recognizing if they might be dealing with an AI-generated identity?
00:36:35.679 --> 00:36:44.400
I mean, I think that it's just we wrote an articles about what to look for on Press Gazette, but I mean, most of the time it's just common sense.
00:36:44.559 --> 00:36:50.559
I mean, a lot of these ones that are out there at the minute, you know, it's just they've made absolutely zero effort.
00:36:50.719 --> 00:36:56.960
I mean, I would say the hallmark that always marks these guys out is that there's no way to book their services.
00:36:57.360 --> 00:37:01.679
So Fiona Jenkins, the gardener, you can't book her as a gardener.
00:37:02.400 --> 00:37:04.719
Like there's no way to contact her to do that.
00:37:04.960 --> 00:37:06.880
Whereas, you know, I'm a freelance journalist.
00:37:07.039 --> 00:37:09.519
It's very easy to find me and book me.
00:37:09.920 --> 00:37:12.000
But with these guys, there's no way to do that.
00:37:12.159 --> 00:37:15.360
And it's like it's it makes no sense because people have to make a living.
00:37:15.519 --> 00:37:22.800
And uh, if you're just a talking head that all they do is comment, there is no way to actually book your services in what you're supposedly an expert in.
00:37:23.039 --> 00:37:24.719
That's an excellent tell, isn't it?
00:37:25.519 --> 00:37:35.760
And so one of the things I'm working on is as we become more and more machinable, if you like, and everything that you have just been discussing is about this world of machinability.
00:37:35.920 --> 00:37:43.679
And actually, if you can't turn humans into machines, you just invent human-like machines that converse and communicate.
00:37:43.920 --> 00:37:48.719
We need to be even more unmachinable in very important aspects of our lives.
00:37:48.800 --> 00:37:53.679
And you've talked about some, you know, with judgment, with adaptability, lots of different areas.
00:37:53.920 --> 00:37:57.039
Where do you think we will net out 10 years from now?
00:37:57.280 --> 00:38:05.039
Will everything be much more machinable and we won't really be able to recognize reality, but we're kind of in it anyway, so we have to go with the flow.
00:38:05.199 --> 00:38:11.280
Or will there have been a backlash and a kind of reversal, if you like, of unmachinable traits?
00:38:11.679 --> 00:38:13.280
I think both things would probably happen.
00:38:13.440 --> 00:38:21.760
There'll be a move towards a sort of very sort of technology-centered, AI-driven world, but at the same time, there'll be a backlash.
00:38:21.840 --> 00:38:39.440
And I think that, you know, things that have been out of fashion, like actually picking up the phone and talking to people on the phone, will come back into fashion because people will want to authenticate, they'll want to approach things in a sort of more human, less screen-mediated way.
00:38:39.599 --> 00:38:51.199
And uh, I think that we'll see a significant kind of wider cultural backlash against AI everywhere and towards a sort of more human, more flesh and blood approach.
00:38:51.440 --> 00:38:54.159
Do you think you'll still be writing about this in 10 years, then, Rob?
00:38:54.880 --> 00:38:57.760
Well, I mean, I have to say it surprised me that uh it surprised me.
00:38:57.840 --> 00:39:00.800
I'm still writing about it a year later and it doesn't seem to be going away.
00:39:00.880 --> 00:39:02.000
So uh who knows?
00:39:02.159 --> 00:39:05.119
If I'm still around in 10 years, then yeah, probably.
00:39:06.880 --> 00:39:07.360
Brilliant.
00:39:07.599 --> 00:39:09.280
It's brilliant to hear the background to it.
00:39:09.360 --> 00:39:13.920
Um, everybody's got to follow your stories because they're absolutely fascinating.
00:39:14.079 --> 00:39:21.760
And uh I think it's fair to say that we're we're populated by probably more virtual AI personas than real people.
00:39:21.920 --> 00:39:28.000
People were worried about a population explosion in the 60s and 70s, but they they didn't realise there was going to be a virtual one.
00:39:29.199 --> 00:39:29.599
Indeed.
00:39:29.920 --> 00:39:30.400
Thanks, Rob.
00:39:30.559 --> 00:39:31.360
Brilliant to speak to you.
00:39:31.440 --> 00:39:31.840
Thanks for your time.
00:39:32.079 --> 00:39:32.400
Excellent.
00:39:32.480 --> 00:39:33.119
Thanks, Tracy.
00:39:36.960 --> 00:39:39.360
My thanks to Rob Waugh for joining me.
00:39:39.519 --> 00:39:45.119
What Rob's investigation shows is that the problem isn't misinformation, it's something far deeper.
00:39:45.360 --> 00:39:47.519
Some people call it synthetic experts.
00:39:47.679 --> 00:39:50.159
I prefer to call it identity slop.
00:39:50.400 --> 00:39:59.039
Because this is the moment when personal identity becomes just another form of highly commoditised and commercialised scalable content.
00:39:59.280 --> 00:40:02.880
Produced really only because the system needs it.
00:40:03.119 --> 00:40:09.280
And that raises larger issues for all of us living in AI-mediated information systems.
00:40:09.599 --> 00:40:13.039
Mechine Dialogues is a special series from the future of you.
00:40:13.360 --> 00:40:17.679
Find us on Spotify, Apple, and YouTube, or at Mechine.com.
00:40:17.760 --> 00:40:24.960
That's mechin e.com, or at tracyfollows.com and search for podcasts.
00:40:25.199 --> 00:40:30.880
We'd love for you to leave a rating if you like the show and do subscribe so you never miss a transmission.
