TRANSKRIPT 🔗
James Dooley: Perception engineering. Today I'm joined with Jonathan Kiekbusch and Jabez Reuben, and we're going to be talking about perception engineering and how to shape what large language models say about your service, product, brand, or business. It's no longer enough just to be cited. You need to start understanding that you could be cited and it could be with a negative sentiment. We need to try to make certain the perception and the sentiment is a positive sentiment, and not only you being cited, but you're actually being recommended by the large language models. Jonathan, please expand upon what I've just said there. What is perception engineering?
Jonathan Kiekbusch: Sure. So my, the, the argument is, as you've just said, that it's well and good being able to influence an AI model to say that, you know, you, that, that you are worthy of being in a list of recommended products or services. Shaping exactly what the use case is for your product or service. Um, making sure that you're the number one recommended one. Making sure that you're being compared on the right axis, if you will. Um, all of these are, um, key elements. And then there's one more, which is essentially reputation management. Right? So, um, making sure that if there was something in your past that, um, detracts from being the number one choice, you want to make sure that, um, you can, you can deal with that. There's a really great example where we were working with a, um, computer brand, and this computer brand creates mini PCs. They had two key things that were, that were holding them back, which is the one was that they had at some point no dedicated hardware towards, um, gaming, and they had hardware failures causing, like, overheating and causing things like that. So what's happening is you would search for, you know, whatever, best mini PC for gaming or stuff like that, and it would surface them. But then it would say in the description about their brand that, like, you want to be wary because they've had these problems in the past. To, to me, that is as bad as not being in that answer at all because it's effectively recommending people not to purchase you. Um, you know, if you're looking for a gaming PC and it's saying it's not fit for gaming and it's overheating, you're probably going to check out. Um, so dealing with that is what we try to do under the perception engineering model.
James Dooley: Do you know what's crazy about the, the amount of people that we get on, me and Jabez, and we speak to, that chases volume and quantity of citations. "Oh, we've been cited this amount of times." That's fine. Are you then going checking the citations against whether it's got a positive sentiment? Because exactly like what you've just said, I'd prefer not to be cited than be cited with a negative sentiment saying don't use these, use these. Like, it's, it's absolutely ludicrous for me. The people that chase the volume of AI citations is like just chasing the volume of reviews, but not bothered if they're all one-star reviews. Like, it's, it's absolutely ludicrous. Volume of reviews doesn't mean positive, like a positive, if they're all one stars, which is exactly what some of them are doing. And they're obsessed by volume of citations. And I'm like, guys, you're completely missing the difference between being cited and being recommended. And this is why Jabez was excited to get you on about perception engineering, to understand that being cited is not enough. Jabez, I'll let you carry on now because I'm sure you've got some questions on it.
Jabez Reuben: So Jonathan, how do you go about fixing the wrong sentiments and wrong analysis? Let's start from there. Let's take the case of this mini computer, uh, mini...
Jonathan Kiekbusch: Sure, sure, sure, sure, sure. So there's, there's... The problem can take quite a few different shapes, right? So the first one is, um, sadly still very common, and that's retrieval. And retrieval is where we identify that actually the information about the product or service is on the website, but you... It can't be retrieved because either they're blocking crawlers or the only way to see that content is, um, by, by having JavaScript enabled, for example. They also haven't optimised their site at all as far as, like, you know, having that information in a schema, yada yada yada. That's the first one. The solution is quite self-explanatory. We, we solve those technical hurdles and we try to, uh, make sure that everything's crawlable, visible, etc. And then we keep monitoring, um, at, uh, various, like, depth to see whether it's actually being, um, uh, crawled and then ultimately surfaced. That's the first thing. The second thing is then corroboration. And corroboration essentially is maybe we're already saying this about ourselves, but maybe there's so much conflicting information out on the internet that it's outweighing it. I have this view, which is that AI isn't adjudicating the truth. It's literally just counting the yeses and the nos and is seeing what's weighing more heavily on the scale. So if we have it just mentioned on our own page, which is obviously going to put us in a good light, and then we have a bunch of content out on the internet that is saying that's not the case, that's probably going to outweigh this one thing that we've said. So in that case, we need to corroborate it by going to every marketing channel and saying, guys, we've got this problem. Our hardware overheats, uh, or it used to overheat. We've now dealt with this problem, and here's what we need you to do. We need you to mention that, you know, you ran this test that makes sure that if it were to overheat, this would be what surfaces it, and it didn't. It's great. We want to get third-party testing done that can be published on pages where we say, okay, these, these, you know, it's passed all these tests. We want to work with, like, platforms that can publish that content, etc. And then the third issue
Load full transcript
is one that we also come across, which is marketers making requests to be shown for things where there's no product-market fit, or simply said, their product doesn't do the thing they want to be surfaced for. So if, going back to something we spoke about in another episode, we go back to the action camera. If your action camera isn't waterproof, you can recommend it as many times as you want for being waterproof. You can probably even get it to show up a couple of times, but the moment that people buy it and they go on Reddit and they flame you for saying your product isn't waterproof, it's going to get outweighed and you're going to lose that position. So those are the, the three sort of core pillars of, of, of how we deal with it.
Jabez Reuben: For corroboration, most of people chase premium sites or premium...
Jonathan Kiekbusch: Tech sites or, you know, whatever your niche premium, uh, source is. Can we utilise mid-tier publications or just smaller platforms or even social media or, or, you know, smaller forums? Can we... Because not everybody has massive big budgets. So, and people have to play smart with whatever resources and, and budgets we have. Can we, you know, fix these after fixing all these issues in your product and now we need to build corroboration and I don't have the budget. My client doesn't have the budget to go on TechCrunch to do a new analysis or new review. Can we use smaller or mid-tier publications, platforms, forums to fix that?
Jonathan Kiekbusch: Yeah, I'm going to give you the old SEO answer of it depends. I think that, like, I've seen... I've seen people, you know, I've, I've been able to influence AI, as I'm sure you all have, with, like, a $14 self-built domain and, and slapping on a Claude website and testing that out. And I'm talking about that in my, in my talk in Chiang Mai as well. But it just depends on the position that you want to take, right? I think you can 100% influence AI by doing just really good marketing and reaching out to micro-influencers who are going to do it just for free product. Uh, going into the forums and just being a passionate member of the forum and doing that. Going to mid-level sites. The one problem that I would say with mid-level sites and, um, using them, which, like, we use them, marketers ruin everything. If there's a good mid-level website that's, that's capable of influencing the LLMs, guess what? Everybody else knows too. And they are going to get absolutely smashed by marketers using them to influence the AI answers. So, you know, back in the day, I'm, I'm sure you've come across this, there used to be the, um, the basically link farm TechBullion, um, for, like, backlinks. And then every single SEO platform started selling links on TechBullion. That's the exact same thing that's going to happen here, right? We're going to find the website that's reasonably priced, that's going to influence everything. Everyone's going to flock to it and guess what? They're going to correct
against it. So yes, you can. The best way to do it is by doing real marketing across all platforms and also using these mid-tier websites, uh, in order to build a moat. And then if you can also get into the big platforms, then I think that will be even better.
Jabez Reuben: So Dooley and I, we recently were doing some testings... just, I think, just day before, I guess, where we started seeing ChatGPT logged-out fan-out queries. So the query was, what do you think about person's name SEO expert? And it started, you know, we, we were just tracking the fan-out queries. And what was very, very surprising and a bit scary as well, that in the very first query, uh, in the very first query fan-out, it would, it would spit out keywords that are so closely related to that particular SEO expert. So, for example, Koray, in the very first query or very first sentence, it would look for holistic SEO. For Dooley, who has been doing so many podcasts, it would, it was straight away searching for recent interviews. And I won't take the name, but we, we know an SEO expert, for him the very first query was controversies. Now, how do you, first of all...
James Dooley: And I think this is part of your perception engineering, you know, specialisation as well. You see all these perceptions that are so, I would say, hard ingrained into LLMs now. How much time would it take for us to see LLMs change perception for a brand? And is, is there a possibility that if it has gone to the level where in the fan-out queries, you, we are seeing these very strong perceptions, it's going to take a very long time to fix those?
Jonathan Kiekbusch: Well, so for my, my perception... My, my perception... My, um, my, my thinking on this, and I'm not the most technical. I have a firm understanding of this and I could be way off field here, but it's, like, for me it all comes down to cost, right? These platforms have got to figure out how to bring down the retrieval cost as much as possible. And the best way to do that is just surface more accurate content with less queries as quickly as possible. So if you search for running shoes, guess what? It's probably going to add Nike and Adidas and whatever to the end of your fan-outs because it knows that these are the, the, the brands that you're most likely going to be searching for. If it just searched for just running shoes or just uphill running shoes, whatever, it would have to retrieve way more content in order to what it thinks answer in a satisfactory way. So, I think that we're going to actually see more of this. And my prediction is that the mid-level of the market is going to get destroyed. If you're really small and niche and local, it's going to be really easy and still, and it is right now, really easy to influence AI to surface you, to give you the perception that you want, all of that. And then if you're a mega company that's got 150 people in their marketing team, tens of millions of dollars being spent on it, it's also easy because AI already thinks that they are the solution. You're just tweaking the, the perception and you're already in, like, the learning corpus, etc. I think where it's going to get brutal is that mid-level of companies that are big from, like, my individual perspective, you know, perception of, like, my own bank account versus theirs, but they are way, way, way too small to compete with the Nike and the Adidas, etc. And for them it's going to get really difficult because eventually the LLMs are just going to answer with the, like, pre-decided products or services. That's, that's a loose prediction. As far as how long it takes to shape that, it doesn't take long at all right now. And I think that's because they're still so thirsty and they're also, um, um, especially for anything that's using live retrieval, that's, that's doing live search. I think there it's, like, you can publish something today and see it cited tomorrow, right? I think what's a little bit harder is if you want to influence what you're saying, where you, where you influence the actual, um, uh, the, the fan-out queries. We need to influence it enough to make sure that it already thinks that we are the solution or this is the thing that we're being associated with. And so for that, I don't know what the time range would be, but I would imagine that would entirely depend on how aggressive you are and how much content that you put out. So, for example, if let's say there's something that really tragic happened and your brand killed someone, like, let's just be really, really dramatic. Like, you had a product failure, you killed someone, a thousand news stories about it, every platform, we've got influencers talking about it, whatever. I reckon that, you know what, within a few months that's going to be in there, if not a few weeks, in the learning data. Like, whenever they release the, the next round of, like, um, uh, essentially, like, base knowledge, I think that there's a high chance it would already be in there. Versus you posting a bunch of, like, positive stuff or whatever, trying to associate yourself with something. I think that might... It, it just depends on the volume, is my guess.
James Dooley: Yeah, I think on, on that, because I've got a few follow-on questions from it. So, obviously on running shoes, you're appending, um, Nike and Adidas because they're huge brands and I get it. Um, it might append, like, ASICS with regards to running shoes and stuff like that. They're having to base that decision on. But when it was, like, an SEO expert, there's not, like, the crème de la crème kind of brands that's there, like Adidas and Nike, and they still got to get that information from somewhere. Now it's, like, you're implying that obviously that this then is prior to query fan-out being even done. This is manipulating the query fan-out terms. So it's got to be part of the, the training data, which could take months, like you said, because not only for it to be picked up,
but then to be bringing it within the training data, and then for them to update the model. Sometimes they don't update the model for two months. So if it's a brand new thing that's just come out, if they're not going to update it for two months, well, it's definitely not going to be there in the training data for the, for the next two months. So, it could be three, four, five months further down the line for, for that to happen to influence the query fan-outs. But in this scenario where Jabez was talking about a specific person and it was showing up controversies, right? And there has been a lot of controversial things about the person. So, it was, like, it was crazy that for me it was bringing back recent interviews. Why? I've been doing a lot of podcasts. For Koray, it brought, like, holistic SEO and semantic SEO. Why? Because he's closely related to that. Sure. So, as part of the training data, if you can influence that part to be that you're a positive, your product is an amazing product, and it starts bringing you back everything about this, perception engineering isn't just about live retrieval. It's, in my opinion, it's almost like manipulating the Google Autosuggest. Yeah. Someone not even knowing what to search, they start to search and go, "Oh, what's this?" And they click onto it and you're manipulating the auto-suggestion of someone searching for the keyword and your brand. It's them bringing your brand back and it's like, "Wow, now this is new." This has only just happened in the last two weeks, um, that we've started to see the query fan-outs change. And it's almost like, okay, prior to query fan-out, we're now going to use our training data for what we think we know about them, and then we're going to then train the model for the query fan-out to then get the information for live retrieval to see what new controversies, or what new recent interviews have I done, and what is Koray talking about recently about, let's say, semantic SEO. It's like, wow. How can you change that perception? What would you say on, on that? Is that just because you mentioned volume, outweighing, and you think it's a numbers game, which then means you're a big advocate of consensus? Is, is that what you think it's going to have to be for this as well? Just volume of consensus of repeating how great you are, which changes the perception or understanding of your brand and your business?
Jonathan Kiekbusch: I, I think so. And I think the quality of... I think it's, like, quantity and quality. So, I could be, I could be totally wrong and there's going to be someone way smarter than me that's going to disagree with this. But the, the, the testing that I've done is just saying that somebody is the best, for example, the best public speaker, does not necessarily qualify that as a resource for AI to, like, base its opinion on. But if you said, "We questioned 500,000 people at 100 events this year and this person was unilaterally the number one recommended speaker
by all of these people." We then had that data fact-checked by Cornell University and we did this, we did that, whatever. That piece of content suddenly ranks much higher in the value because of the perceived value that that data brings. The same exact thing applies for awards. If you say that you just won this award versus you won this award because you were voted by 100,000 people in your local area who know about the product and service. So I think that quality and quantity make, um, make a difference. And the other thing is, like, I think about this very much in a similar way as sort of, like, the, the concept of, like, entity SEO and basically having these two entities, one of them being the person or the product or the brand and the other thing being the thing that they want to become known for, and creating enough evidence that these two things are connected for long enough so that, um, so that, that it's, like, unarguable. It's unarguable that Adidas equals running shoes or trainers or whatever, right? But they're probably not known for rocking horses. Like, so it's, it's just... I think, yeah, I just think it comes down to, like, the quality and quantity and making sure that we create, like, a consistent brand image or entity image or whatever we want to call it that AI can learn from, and then doing that over time. I think it's also something that will probably be easier for somebody to achieve that's been working on something for a while. Like, you look at, you look at, like, I don't know, Gary Vee is a good example, right? He's been creating, like, a thousand pieces of content a month for, like, as long as I've been alive, basically. And then, you know, I could start doing the same thing, and I think I could make a small wave in some areas, but he's still going to outweigh me with the quantity. So, I don't know. That's... It's, it's a very interesting topic.
James Dooley: Jonathan, one more thing. I think we covered a bit about it in the previous interview, but most of the answers come with categories. So, for example, you search for best cameras, and then you will say... You will get answers for this is best action camera, this is best overall, this is best for waterproof, and so on. Now, since you are in the e-com niche and you may probably be working with brands that are fairly new, not in anywhere in the recommendations or even in the fan-out shortlist as well, how do you force now or plan to get them in those categories? Do you from the get-go plan that we will mark this product or highlight this product as best overall? Do you, or do you pick one of the categories that are showing up in and then say, okay, I will not go for the best overall category, I will go for the best waterproof category? Or do you create a totally new category and then start highlighting, okay, this camera, I will mark it as the best design, which is not showing anywhere, and I'll just keep highlighting it over and over again as best design or best resolution? How do you plan that?
Jonathan Kiekbusch: Yeah, that's a, that's a really great point. So we, um, first of all, I have to say we, we try to bet on winning horses. So we try to work with companies that are already established. Works a lot easier. They tend to have better budgets. So, um, no, the, the, the way that we look at it is... I, I think that the overall best has got some value to it simply because if you're being shown as, like, the overall best, there is a chance that you also show up for the, the sub-prompts or whatever you want to call them, right? The, um, the conversion question is what bothers me with that. The, the thing that bothers me with that is, like, how many people are searching in AI specifically for just best action camera versus searching for, I'm a horse rider and I want to film my horse-riding trips and whatever, that blah blah blah blah blah. And then the fan-out of that is actually, like, best action camera for drops, best action camera for horse riding, best action camera for waterproofness or whatever. And so when we do this analysis, we try to look at the frequency at which the categories are surfaced and try to evaluate whether that indicates any sort of, like, priority. Um, and so then we can, like, invest more heavily in, um, establishing ourselves for those particular attributes or categories or, or, um, axes on which these are being reviewed. I think also there are cases where, especially in e-commerce, where you can actually make a single product or a, a product line fit quite a few different scenarios if you want to, if you, if you will. So, like, the same action camera that's being sold for the horse rider is also being used for the surfer. It's also being used for automotive or whatever, right? And so in SEO you would just create collection pages for those. And I think that there is a world where you can actually make the same product show up for various different category terms, um, quite effectively.
James Dooley: And can I reframe the question? Um, so Jabez there is talking about category framing, and I always talk about the claim, frame and prove model. So you claim something, you frame in a specific way. So it could be, in this scenario we're using action cameras, so best action camera for horse riding, best action camera for hiking, and stuff like that. And then you prove that it's the best by statistics and, and building consensus. So the claim, frame and prove model. And that is great. However, when we're talking there that Jabez was talking about category framing, within SEO, what you would normally do, or what a lot of people would normally do, is it works through the traffic tiers. So they'd start with easy-to-rank-for, low-keyword-difficulty type of terms. We'll start to win on that and then move to the next category, and move up to the next category, and so on and so on. I'm going to have to do the longer tails. So instead of just saying best overall action camera, which might be
difficult and have a lot of competition, it would go for best action camera for hiking, best action camera for, and then another term. Would you, or do you, when you're looking at category framing, think of it working through the traffic tiers? Or think of it as working through, okay, let's try and win the secondary longer-tail keywords, let's do the third-party corroboration on that, and the more we win on the secondary longer tails, the more then in time it's going to win the best overall, if that makes...
Jonathan Kiekbusch: I understand, yeah. So we, we do this in two different ways and we have, um, almost, I would say, like, borderline equal results. So we do that. We create the best... Like, let's say we're doing a listicle. We create one that is for the best action camera and then within that we may put supporting language for our product that says this is specifically good for these, like, scenarios or use cases. We found that to be quite effective. And then equally, um, saying that our, um, action camera for, for hiking is the best one that was tested also supports the overall goal of becoming, like, the recommended brand across the whole category. I think that that's... I understood your question. Yeah. Because on that, just, just, uh, it's not... We don't really do e-commerce and AEO for e-commerce, but for service-based type businesses, what we try to do on the framing part is we need to understand that for us, our ideal customer persona and who we're not built for. So let's say you're saying that you don't generally work for small e-commerce brands that got low budgets. You generally want an enterprise e-commerce brand. When you know that, you might be like, anyone that wants e-commerce SEO or e-commerce AEO that's got under a £1,000 budget, you're going, we can't work with them. So what then you could start to do is doing comparison articles of your brand versus another e-commerce SEO brand. And you could start to say for under £1,000, you can literally then start to say they're the better company. However, if you're an enterprise brand, we're the better company. So you're framing it in a way on a comparison that you don't win every single one, but every one that's your ideal customer persona that you win on. So for action cameras, there might be six things that you win on and two where you think, do you know what? We're never going to beat GoPro for that, or we're never going to beat these in that category because that's just what they focus on. Yeah. Let them win those two categories, but we're going to win these six categories. Yes. Which are longer tail. Is that some things of what you do on third-party sources as well?
Jonathan Kiekbusch: Yeah, I, I think, yeah, yes. The, the really... The really strong thing on this is that I actually have seen strong indication that reviews that are, um, balanced, to where you give something up on the number one position. Like, you say it's great for all of this stuff, but here's one flaw or
here's one thing that's still working out or whatever. Gets cited at a much higher frequency than a piece of content that's just glazing the number one and saying, like, they're the best, they're God's gift. And so we, um, when we try to do that, we do exactly what you just said. We try to identify something that's either a, um, uh, something that's not great but not a deal breaker. Um, or, like, for example, uh, I don't know, you might say it's an amazing product. It's so great for everything, but it's currently not available on Amazon. It's only shipping through their own website. Great. We've said something that's not great about the product, but it's actually not a deal breaker because a person that wants it is probably still going to purchase it, and they're actually not on Amazon. But on the, um, all... What we try to do is do exactly what you just said. We try to identify something that is a category that's not important to us because we don't find that we sell well to that audience, and then we, we give that up to the, to the others. Yeah. So on there, I'm presuming stuff like, let's say, the lighting on the... I don't, I don't do e-commerce. So, like, but let's say the lighting, um, for the action cameras, they don't have, like, a really high-beam light or something like that. You could start saying stuff like, uh, yeah, if you're, if you're a cave night swimmer and, and lighting's important, then we're not for you. And, like, some people go, well, I'm not a cave night swimmer, so yeah, I'll do it for horse riding during the daytime because I'm not lighting. Exactly. Yeah, that's exactly... That's exactly right. For anyone with the... Go on, Jonathan. Good. Have you played around, uh, like, or just random test where you included fake brands in listicles?
Jonathan Kiekbusch: Um, so this is actually something that we're, that we're looking into so much at the moment because there's this argument of, like, how much does your article already need to agree with the consensus that AI has made? And, you know, do you want to include brands that are included somewhere in the data set that you're collecting? Are you not? We're testing it out and we've seen all kinds of things working. Um, I, I can't say that I've found one thing to work particularly worse than others. And I'm saying it specifically like that because I feel like, in a weird way, AI still doesn't quite get it. You know, like, it, it, it just... We've not found one thing where, like, well, that just didn't work whatsoever. So, I, I can't say that I've got conclusively the best solution yet. I think not making it up also kind of helps us because we do have to send a lot of the stuff to our clients, and they're like, "What is this? I've never heard of this. It's not Insta360. It's not DJI. It's not GoPro. Where's this company come from?" Um, but, uh, but yeah, no, there's so much fun stuff you can do now, right?
James Dooley: And do you think AI right now is able to connect the dots or
not? Giving some context, we've been... James has been doing a lot of testings with King of AEO, King of GEO, and then a bunch of other guys who also started doing. And we have a testing group where there's another guy trying to outrank and win for King of GEO award, and he's not been able to do that. But anytime you search King of AEO, King of GEO award, he gets cited. But if you remove the award phrase, he doesn't get cited, which is pretty much the same thing. Yeah. Well, because when the answers come out of who is the King of AEO or GEO, it will give the disclaimer that, okay, James Dooley and XYZ other people are, you know, the recommendations, but also this is a fun game that was done or a fun experiment that was done with an award and all that stuff. Do you think LLMs are able to join the dots to figure out, you know, brands, different products? Or do you think, you know, right now LLMs are not able to do that properly, especially in e-com where for one company maybe one line of product category is great, great reviews, but the other line failed? Does that impact their overall, uh, brand perception across all categories?
Jonathan Kiekbusch: I think that they're trying, but I don't think that we're, um... I don't think that they're doing a good job at it. And a good example of this was, um, what's his name? Metehan's study where he published a piece of a website with, like, 60,000 scientific studies that he made up. And not only did the LLM start citing it across the whole internet, but I think a couple of universities started citing it in their research, which is really, really, really concerning.
Jabez Reuben: Yeah. Really, really worrying. Um, the, the, the thing that I think is happening... The first thing is anybody that takes part in a game like that that Dooley is in does not know the ambition and the work ethic that this guy has. And I think that the, the simple reason why he's not being dethroned is because he probably has a million pieces of content out there placing him as number one. So...
Jonathan Kiekbusch: I think, like, this, this is a, a fool's competition to try and, to try and attempt that. But the second thing is, like, I definitely think they're trying, right? They're trying to build this, like, semantic web of, like, everything about the brand and try to weigh it up. And I think that these things are going to connect, and that brings us back to the fan-out queries. It brings us back to all of this. And this is why I think that there's so much opportunity right now for people to also, like, buy that attention, right? So if you're in really, really competitive niches, you can buy the forums, you can buy the YouTube channels, you can whatever. And, like, we've seen this, right? Like, even in the SEO space, like, there were the big consolidators have been buying everything up, and Ahrefs has bought up Detailed.com, and Semrush has bought, um, a bunch of courses and stuff. And I think they're doing that because they're not stupid and they know that they can control the, the, the learning corpus, which is going to continue to associate them stronger and stronger with the things that they want to be known for.
James Dooley: Perfect. Yeah, for sure. Anyone who's... Anyone who's listening to this, we hope you like the episode on perception engineering. This is more than just doing AEO to get more citations. To get more citations in ChatGPT, to get more citations in Google AI Mode or Google AI Overview. This is actually changing the perception and the sentiment value of your business, your brand, or in this case your products for an e-commerce brand. Jonathan Kiekbusch, Jabez, it's been an absolute pleasure. Great seeing you guys. Great setting.