WEBVTT
00:00:02.160 --> 00:00:15.679
So it's been really interesting because I think over the last, what's it been, six or twelve months, I've gone from really spending a lot of my time working with AI to working with a lot more companies helping them understand how to use AI.
00:00:15.839 --> 00:00:19.760
That's what the last couple episodes have been around culture and things like that.
00:00:20.000 --> 00:00:29.039
But one of the things that I've noticed, as we have the plausible noise problem and all these other things, is that look, there's a lot of people that are using AI to get faster.
00:00:29.120 --> 00:00:31.039
And that's not the problem.
00:00:31.199 --> 00:00:34.240
I think in a lot of ways, that's actually probably the symptom.
00:00:34.479 --> 00:00:41.039
I think that the real problem that I see is that so many people are asking it questions that they already know the answer to.
00:00:41.200 --> 00:00:47.840
They've turned probably one of the most powerful thinking tools in history into just a yes machine.
00:00:48.159 --> 00:00:52.159
And look, and I think it's making your ideas weaker, it's making your decisions softer.
00:00:52.399 --> 00:00:55.520
And for a lot of people, I think it's just making them lazier.
00:00:55.840 --> 00:00:57.840
And I think that's the issue, right?
00:00:57.920 --> 00:01:02.320
Is I think for so many people, I'm not even sure that they know this is happening.
00:01:02.560 --> 00:01:15.920
So that's what I want to talk about today is I want to talk about how you need to stop asking AI to disagree with you, to recognize that it is often a yes man that will tell you, no matter what you're doing, it's a great idea.
00:01:16.239 --> 00:01:19.519
So, welcome to episode 145 of The Crazy One.
00:01:19.599 --> 00:01:21.120
As always, I'm your host, Stephen Gates.
00:01:21.280 --> 00:01:27.120
This is the show where we talk about creativity, leadership, design, and everything else that matters to creative people.
00:01:27.280 --> 00:01:28.640
Now, as always, appreciate it.
00:01:28.719 --> 00:01:34.480
Leave a review, tell a friend, hit the subscribe button so you can once again get the new shows whenever they're coming out.
00:01:34.719 --> 00:01:41.200
Because thanks to AI and my new writing partner, which is Claude, I've got the next 10 episodes already written.
00:01:41.439 --> 00:01:51.120
I finally have got a better handle on what for me have been the two most time-consuming parts of the show, which are writing the episodes and then handling all the content production afterwards.
00:01:51.280 --> 00:01:57.040
So I found a better way of doing it that's going to let me put out content on a much more regular basis.
00:01:57.519 --> 00:01:59.359
But but let's talk about this, right?
00:01:59.519 --> 00:02:12.560
Because I think that in so many ways, it's a trend that I started to notice probably about like two years ago, where and it's honestly one of the reasons why I left Chat GPT because it just was so annoying to me that it was just such a yes man.
00:02:13.039 --> 00:02:22.000
And this is the thing, is that I think right now you sort of see like AI adoption is everywhere, but I think so much of AI literacy is still in such a primitive state.
00:02:22.240 --> 00:02:28.879
Because I think, look, most people have learned it sort of the way they almost treat it like a search engine, right?
00:02:29.120 --> 00:02:32.560
Write this, summarize this, give me an idea.
00:02:32.800 --> 00:02:36.000
But they're prompting it for an output.
00:02:36.560 --> 00:02:37.840
And I get it, right?
00:02:37.919 --> 00:02:46.159
Like again, if you come from search and a lot of the other ways we've used technologies that sort of looked similar, that was this the way you would do it.
00:02:46.240 --> 00:02:52.400
And so I think that's why like this habit got baked in early, and there aren't that many people that are really questioning it.
00:02:52.719 --> 00:02:58.639
But I think that's the thing that I'm seeing is that AI has just become this like confirmation machine.
00:02:58.879 --> 00:03:04.240
It reflects back what you're already thinking, but somehow it's it's doing it faster and it sounds more confident.
00:03:04.960 --> 00:03:08.960
I was watching an episode of last week tonight, I don't know what, a week or two ago.
00:03:09.120 --> 00:03:17.360
There's a whole story on there on AI chatbots, and how I think it was Chat GBT had convinced a man that he'd come up with a totally new form of math.
00:03:17.520 --> 00:03:21.199
He'd discovered all this new sort of thing, and he would prompt it and say, like, are you sure?
00:03:21.360 --> 00:03:23.120
And it would say, Yes, like you're a genius.
00:03:23.280 --> 00:03:24.639
You've come up with this thing.
00:03:24.960 --> 00:03:28.719
Well, of course, in the end, none of that was actually real.
00:03:29.520 --> 00:03:31.120
But I think that's the thing, right?
00:03:31.199 --> 00:03:37.360
Is it it's that confidence with which it answers that's the dangerous part.
00:03:37.840 --> 00:03:50.319
Because again, I think it it says things like it's the authority, whether it's right or wrong, whether you understand the sources it's citing are good or bad, like whether it's like it's just it's giving you this false sense of confidence.
00:03:50.560 --> 00:03:55.439
And I think that's really the step before plausible noise, right?
00:03:55.520 --> 00:04:00.000
That's why people feel confident to do everybody else's job, to do these sort of things.
00:04:00.400 --> 00:04:02.400
And and that's the problem, right?
00:04:02.479 --> 00:04:04.560
Is because everybody's like, oh, it must be right.
00:04:04.639 --> 00:04:06.080
It told me that.
00:04:06.560 --> 00:04:08.240
But here's the thing that I've learned.
00:04:08.479 --> 00:04:18.480
If you aren't actively pushing back, if you aren't using it almost like a sparring partner, then you're getting almost your own assumptions dressed up in better language because it's a learning model.
00:04:18.560 --> 00:04:22.639
It learns what you want, what you like, and what you want.
00:04:23.680 --> 00:04:28.800
And I think that that's where for me, like I said, I try to use it almost like a sparring partner philosophy.
00:04:28.879 --> 00:04:30.319
This is how I've always treated it.
00:04:30.480 --> 00:04:41.600
And I think it really changes everything that you get out of it, and that you can start to get that force multiplier without losing all the best parts of what makes us all creative and special.
00:04:42.240 --> 00:04:49.279
Because I think look, at the end of the day, I think in a lot of cases right now, AI is just going to tell you what you want to hear, right?
00:04:49.439 --> 00:04:51.120
It's trained to be helpful.
00:04:51.279 --> 00:04:54.959
And I think in too many cases, helpful means it's trained to agree with you.
00:04:55.199 --> 00:04:58.000
So you, if you ask it like, hey, is this a good idea?
00:04:58.160 --> 00:05:01.439
It will find a reason to tell you why it is.
00:05:01.839 --> 00:05:11.600
Again, there was Ryan Sirhans, a big um real estate broker in New York, said he lost a deal because of Chat GPT, because the seller went in and said, Am I getting enough money?
00:05:11.680 --> 00:05:12.560
And it said, No, you're not.
00:05:12.720 --> 00:05:13.680
Here's the comps why.
00:05:13.759 --> 00:05:15.759
And the buyer went in and said, Am I paying too much?
00:05:15.920 --> 00:05:17.279
And it said, Yes, you absolutely are.
00:05:17.360 --> 00:05:18.720
Here are the comps why.
00:05:18.959 --> 00:05:24.480
So both sides, empowered by Chat GPT, were convinced that they were right and the deal blew up.
00:05:25.120 --> 00:05:28.319
Again, I think if you ask it, you know, is this a good idea?
00:05:28.560 --> 00:05:37.920
And I think if you ask it even how will this fail, it will find those reasons too, because like the model doesn't have conviction.
00:05:38.480 --> 00:05:39.040
You do.
00:05:39.120 --> 00:05:47.120
So I think that like the output quality isn't always necessarily the problem, but it's it's questioning that quality that is.
00:05:47.839 --> 00:05:58.319
And I think a lot of people have fallen into this bad habit where they're sort of use it to validate what they're already thinking, as opposed to like really using AI to find the weak spots.
00:05:58.399 --> 00:06:10.079
And I think that's always been my approach, which is why I fell into it with AI, is once I come up with an idea, I want to use research or data or testing or whatever to break it.
00:06:10.319 --> 00:06:11.600
I want to find the weak spots.
00:06:11.680 --> 00:06:19.040
I want to find how it can be better, as we're with too much research and too many other creatives, they're going out there to find validation, right?
00:06:19.120 --> 00:06:22.000
They're going out there for somebody to tell them it's a good idea.
00:06:22.240 --> 00:06:27.360
And now with AI, it's created a smaller echo chamber for that to happen.
00:06:27.759 --> 00:06:38.160
And I think, you know, for me, that's where it's like we need to start to think through like, how do you work with the smartest, most challenging person you know?
00:06:38.560 --> 00:06:40.160
We we all have worked with them, right?
00:06:40.240 --> 00:06:42.959
Like that person that's just like they always seem to have the answer.
00:06:43.040 --> 00:06:46.000
They can see it from another angle, they're always gonna challenge what you're thinking.
00:06:46.160 --> 00:06:53.120
The best leaders, the best CEOs, the best, like kind of again, real thinkers I've ever worked with, this is what they did, right?
00:06:53.199 --> 00:06:56.720
And and you wouldn't just present them with an idea.
00:06:57.040 --> 00:07:00.879
You'd go in and know that it would be a debate or an argument that they were gonna push back.
00:07:00.959 --> 00:07:06.160
You had to defend your position or revise or see where there were holes in it.
00:07:06.399 --> 00:07:12.720
And I think that AI can be that way, but only if you set it up to be that way, right?
00:07:12.879 --> 00:07:14.879
Because look, the tool doesn't change.
00:07:15.040 --> 00:07:18.319
I think that your posture towards it needs to.
00:07:18.879 --> 00:07:25.360
Because for me, any idea, anything where you've done the thinking always survives that fight, right?
00:07:25.439 --> 00:07:26.959
It survives the pushback.
00:07:27.279 --> 00:07:34.560
And in general, I think this is just a culture we need to build more of, where I've talked about we need to kill status meetings and start having decision logs.
00:07:34.800 --> 00:07:36.959
This is why this needs to happen.
00:07:37.199 --> 00:07:41.759
Because look, I think that like every strong idea will survive that fight.
00:07:42.079 --> 00:07:46.240
And AI, in a lot of cases, I think is is one of the easiest.
00:07:46.399 --> 00:07:53.040
I don't know if I'd say safest, I'm not sure that feels right, but it's one of those places to have that fight before the room does it for you.
00:07:53.199 --> 00:07:57.519
To start to see where are your blind spots, where are the weak points in sort of doing this.
00:07:57.759 --> 00:08:05.519
And I've developed what are really kind of four different types of what I'll call like challenge prompts.
00:08:06.000 --> 00:08:08.720
And and this is again where I'm going to go back.
00:08:08.879 --> 00:08:13.199
One, I always show up with an idea, but then I want to find the holes in it.
00:08:13.439 --> 00:08:16.879
So there are four different sort of ways that I've tried to do this.
00:08:17.199 --> 00:08:19.839
The first one is what I'll call like a failure prompt.
00:08:20.000 --> 00:08:24.639
And that's where I what I'm gonna do is I want it to make it argue with me, right?
00:08:24.879 --> 00:08:27.199
By asking it, like, look, how's this gonna fail?
00:08:27.279 --> 00:08:30.399
What would the most skeptical person in the room say?
00:08:30.639 --> 00:08:33.840
Because what I want to do is I want to force it off that happy path.
00:08:34.080 --> 00:08:37.120
I want to force it off that, like, yes, like Steve, you're brilliant.
00:08:37.200 --> 00:08:37.840
This is great.
00:08:37.919 --> 00:08:39.279
You're gonna go change the world.
00:08:39.440 --> 00:08:41.519
Because can't I just I don't need that shit, right?
00:08:41.600 --> 00:08:42.960
Like, I just that's not what I'm looking for.
00:08:43.039 --> 00:08:44.720
That's not gonna help me in my work.
00:08:44.960 --> 00:08:49.519
So again, I think that sort of thing of like, look, what's my biggest critic gonna say about this?
00:08:49.919 --> 00:08:52.720
Forces it down a different line of thinking.
00:08:52.960 --> 00:09:00.720
I think the next one for me is to really try to try to find prompts that will expose assumptions.
00:09:01.360 --> 00:09:06.639
Because look, I think that like, how do I surface and find what I'm taking for granted?
00:09:06.879 --> 00:09:10.559
So this is asking at things like, what does this idea assume is true?
00:09:10.799 --> 00:09:13.039
What would have to be true for this to work?
00:09:13.200 --> 00:09:13.440
Right.
00:09:13.519 --> 00:09:16.799
Because look, I think most ideas don't fail on the surface.
00:09:17.039 --> 00:09:30.720
They fail because of like a buried assumption that either everybody misses or that nobody checked, or that there is something in there where it's like, you know, this is gonna be great, but it's gonna die in legal or tech or like somewhere else, right?
00:09:30.799 --> 00:09:39.039
Like something that I'm going to assume that they can do it and that it all sounds very like magical and great to go pitch, but it's not gonna get out the door.
00:09:39.840 --> 00:09:51.759
I think that a lot of it also for me is that again, whenever it states things back with such confidence, the third thing I want to do is really sort of just have prompts that make it prove itself and really challenge its data validity.
00:09:52.080 --> 00:09:55.519
In asking things like, again, how confident are you in this and why?
00:09:55.679 --> 00:09:59.200
What's the strongest argument that, you know, in what you just told me?
00:09:59.360 --> 00:10:01.759
What are the actual data sources that you are citing?
00:10:01.840 --> 00:10:16.159
If you are building your own models or your own projects or doing those sort of things, to bake that into the instructions, that I only want you to be able to find like legitimately, like legitimate data sources that can be vault that can be verified from multiple different directions.
00:10:16.320 --> 00:10:25.039
And that again, I don't want you finding like just kind of one crackpot's blog who quoted a bunch of stuff and then just assuming that that's the right data.
00:10:25.279 --> 00:10:34.080
So I think this is also what I built into a lot of the models and a lot of the things that I work with, are where it's going to start to know that it needs to argue with me, not to agree with me.
00:10:34.159 --> 00:10:35.440
It's going to question those assumptions.
00:10:35.519 --> 00:10:38.960
It's going to show me what the data is that it's doing.
00:10:39.200 --> 00:10:49.120
And then the last thing is, and again, this is a concept that I've talked about in past shows, is I'm I want it to really almost start to run pre-mortem prompts.
00:10:49.360 --> 00:11:02.320
So this is like, right, like run the failure scenarios, run those sort of things that, you know, again, so it will say, like, look, this is a great idea, but like it's going to fail 12 months from now, and here's why.
00:11:02.639 --> 00:11:02.960
Right?
00:11:03.039 --> 00:11:09.360
Like, again, what's it'll start to, I'll start to ask it, like, look, what's the most expensive that we mistake that we could make here?
00:11:09.759 --> 00:11:12.320
And that's borrowed from product thinking, right?
00:11:12.399 --> 00:11:14.799
That's not just good practice in AI.
00:11:14.960 --> 00:11:19.279
That's actually just good product thinking and good systems thinking.
00:11:19.519 --> 00:11:23.600
But again, to start to run the pre-mortem prompts of like, what are the guardrails we should have?
00:11:23.759 --> 00:11:25.519
What are the like what's the biggest consequence?
00:11:25.679 --> 00:11:26.720
What's the blowback?
00:11:26.879 --> 00:11:29.200
Can we claw this back if this goes wrong?
00:11:29.360 --> 00:11:29.519
Right.
00:11:29.679 --> 00:11:30.720
There's those sort of things.
00:11:30.879 --> 00:11:35.600
And I think I've got an episode coming up that just focused on pre-mortems.
00:11:36.000 --> 00:11:38.559
I'm thinking now that probably should be the next episode.
00:11:38.799 --> 00:11:44.480
But for me, it's that like it's the failure prompt, the assumption exposure, data validity, and pre-mortems.
00:11:44.720 --> 00:11:50.320
Those are the sort of things that you need to be getting it to do, baking it into your models to push back.
00:11:50.559 --> 00:11:50.879
Right.
00:11:50.960 --> 00:11:55.600
And I think that in a lot of cases, like it's making this a habit, not just a checklist.
00:11:55.919 --> 00:11:58.639
Because this only works if it becomes part of the way you think, right?
00:11:58.720 --> 00:12:02.399
Not a step that you remember at the end, like, oh, did somebody ask legal?
00:12:02.639 --> 00:12:07.919
But before, right, like you take any AI output seriously, ask what it got wrong.
00:12:08.080 --> 00:12:11.440
Before you act on any AI-assisted idea, right?
00:12:11.600 --> 00:12:12.879
Run the pre-mortem.
00:12:13.039 --> 00:12:17.519
Before you present anything built with AI, stress test the assumptions.
00:12:17.840 --> 00:12:21.840
And again, my thing isn't that like it's to distrust the AI.
00:12:22.399 --> 00:12:30.799
It's it's in using it in a way that you would with any smart colleague, with any sparring partner, with any great partner you've ever had, right?
00:12:30.879 --> 00:12:32.960
They don't just take it at face value.
00:12:33.120 --> 00:12:38.159
They push you, they push your thinking, they don't just take the first answer as gospel and agree with you.
00:12:38.320 --> 00:12:44.000
The weakest organizations and the worst leaders, I've talked about on here for a freaking decade, right?
00:12:44.240 --> 00:12:47.440
Are the ones that surround themselves with people who disagree with them.
00:12:47.679 --> 00:12:49.679
It's that yes people culture, right?
00:12:49.759 --> 00:12:55.679
Like everybody on my team thinks I'm brilliant, but somehow like our ideas aren't going anywhere, our stock price isn't up, right?
00:12:55.759 --> 00:12:56.879
Like that's why.
00:12:57.600 --> 00:13:06.879
And look, and I think that here's the harder truth in all this is that I think the reason people don't do this isn't that they don't know how.
00:13:07.440 --> 00:13:09.120
It's that they don't want to.
00:13:09.759 --> 00:13:10.159
Right?
00:13:10.240 --> 00:13:12.559
Like confirmation feels good.
00:13:12.720 --> 00:13:14.960
Like getting validated feels comfortable.
00:13:15.120 --> 00:13:21.519
Saying, oh, look how smart I am and how well I'm using AI, and I'm a genius, and I'm out on the frontier of this, right?
00:13:22.399 --> 00:13:24.639
That feels better.
00:13:25.279 --> 00:13:29.120
But my thing is at the end of the day, that's a short-lived, short-sighted approach to these things.
00:13:29.200 --> 00:13:33.360
Because look, if your idea survives the challenge, then you can present it with real confidence.
00:13:33.519 --> 00:13:40.080
If you're somebody that isn't great at research and that's not what you like has always been your strong suit, then this is a great tool to be able to do this.
00:13:40.320 --> 00:13:41.440
Because that's the thing, right?
00:13:41.759 --> 00:13:48.480
If it doesn't survive the test, you just saved yourself walking into a room and getting destroyed, right?
00:13:48.639 --> 00:13:51.919
Of presenting something that is going to erode what you are doing.
00:13:52.000 --> 00:13:59.600
And that's why I said, like, so many people seem to be on this road to destruction, but they're so happy about it because they're feeling validated and they're doing it quickly.
00:13:59.840 --> 00:14:03.360
But if you're doing that, look, at the end of the day, either way you win.
00:14:03.600 --> 00:14:11.519
Because like if you're willing to have that fight first, if you're willing to test that, if you're willing to get into that habit of making your ideas stronger, your work just gets better.
00:14:12.080 --> 00:14:13.200
And that's the thing, right?
00:14:13.279 --> 00:14:21.600
The people who are going to be the most impactful, the most dangerous, the most the ones who sort of really excel, they're not going to be the ones that are using the best tools.
00:14:21.840 --> 00:14:28.399
They're going to be the ones that are asking the best questions, that are building the best models, that are finding the best workflows.
00:14:28.559 --> 00:14:33.279
And that's why I keep coming back to like this is AI is not a technology problem, right?
00:14:33.360 --> 00:14:41.840
It's a culture problem, whether it's the culture of a company, the culture of a team, or your individual kind of way of working in that kind of like microculture.
00:14:42.639 --> 00:14:43.919
Because that's the thing, right?
00:14:44.000 --> 00:14:52.879
Like take the idea you're working on, the one you're doing right now, or why the one that maybe you're in Claude or ChatGPT working on as you're listening to this show right now, today.
00:14:53.039 --> 00:14:57.200
And before you do anything else with it, run through these prompts, right?
00:14:57.360 --> 00:15:00.559
Do the failure, the assumption, the validity, and the pre-mortem.
00:15:00.799 --> 00:15:02.320
Those four rounds, right?
00:15:02.480 --> 00:15:05.120
It's going to take you 15 minutes tops.
00:15:05.600 --> 00:15:06.879
See what survives.
00:15:07.039 --> 00:15:08.080
See what doesn't.
00:15:08.240 --> 00:15:11.759
Because look, whatever's left after that fight is going to be worth your time.
00:15:11.919 --> 00:15:16.320
And that you being comfortable and validated is not going to get you where you need to go.
00:15:16.480 --> 00:15:17.519
I know it feels better, right?
00:15:17.600 --> 00:15:18.879
Like, oh, you're great.
00:15:18.960 --> 00:15:29.120
And and like, look, I mean, you can tell that there are certain engineers of a certain generation that just program things in a certain way, focusing on that validation.
00:15:29.600 --> 00:15:30.559
But that's the thing, right?
00:15:30.720 --> 00:15:33.600
AI is not going to make you smarter by default.
00:15:33.919 --> 00:15:34.240
Right?
00:15:34.320 --> 00:15:39.519
It's not going to make you faster if like than a level of thinking that you're already doing.
00:15:39.840 --> 00:15:40.159
Right?
00:15:40.320 --> 00:15:45.759
You have to decide to think harder, to be able to hold yourself to a better standard.
00:15:45.840 --> 00:15:47.120
Then you use the tools.
00:15:47.279 --> 00:15:49.120
Then you go in and do this.
00:15:49.360 --> 00:15:51.279
That's how, again, you want to stay ahead of it.
00:15:51.360 --> 00:15:52.879
That's how you don't become commoditized.
00:15:52.960 --> 00:16:00.879
That's how we talked about in the like the last episode, where you're starting to move up into those top two quadrants where you are more of a strategist.
00:16:00.960 --> 00:16:01.919
You are more of a thinker.
00:16:02.000 --> 00:16:06.559
You are doing more of those sort of things where, again, it's not just taking it at face value.
00:16:06.639 --> 00:16:08.159
Because that's where I see most people screw up.
00:16:08.240 --> 00:16:14.080
It's this copy and paste version or like Google search version of using AI.
00:16:14.399 --> 00:16:17.120
And that again, think of it like a sparring partner.
00:16:17.279 --> 00:16:21.200
Think of it like that best partner, the person that challenged you the most.
00:16:21.360 --> 00:16:26.960
Because if you can get it to do that, then you can really start to have a big unlock in the way that you're working.
00:16:27.039 --> 00:16:28.960
That's going to be a big help.
00:16:29.200 --> 00:16:30.480
But I'm also curious, right?
00:16:30.559 --> 00:16:41.120
Like as always, if there are ways that you're working with it, if there are tricks that you've found, if there's other techniques that you found is sort of pushing back, if there's instruction sets that you've done or models that you built, or public ones that you want to share, let me know.
00:16:41.200 --> 00:16:42.799
I'm happy to share them out to people.
00:16:42.960 --> 00:16:44.000
But that's the thing, right?
00:16:44.080 --> 00:16:48.720
Is that we have to start normalizing this sort of thing to be able to get things to be better.
00:16:49.039 --> 00:16:51.679
So as always, thanks for your time.
00:16:51.840 --> 00:16:52.799
Thanks for listening.
00:16:53.039 --> 00:16:53.919
I hope this helps.
00:16:54.080 --> 00:16:58.879
I hope that it helps you think through your work a little bit differently and try those things a little bit differently.
00:16:59.039 --> 00:17:02.399
And hey, as always, stay crazy.
00:00:02.160 --> 00:00:15.679
So it's been really interesting because I think over the last, what's it been, six or twelve months, I've gone from really spending a lot of my time working with AI to working with a lot more companies helping them understand how to use AI.
00:00:15.839 --> 00:00:19.760
That's what the last couple episodes have been around culture and things like that.
00:00:20.000 --> 00:00:29.039
But one of the things that I've noticed, as we have the plausible noise problem and all these other things, is that look, there's a lot of people that are using AI to get faster.
00:00:29.120 --> 00:00:31.039
And that's not the problem.
00:00:31.199 --> 00:00:34.240
I think in a lot of ways, that's actually probably the symptom.
00:00:34.479 --> 00:00:41.039
I think that the real problem that I see is that so many people are asking it questions that they already know the answer to.
00:00:41.200 --> 00:00:47.840
They've turned probably one of the most powerful thinking tools in history into just a yes machine.
00:00:48.159 --> 00:00:52.159
And look, and I think it's making your ideas weaker, it's making your decisions softer.
00:00:52.399 --> 00:00:55.520
And for a lot of people, I think it's just making them lazier.
00:00:55.840 --> 00:00:57.840
And I think that's the issue, right?
00:00:57.920 --> 00:01:02.320
Is I think for so many people, I'm not even sure that they know this is happening.
00:01:02.560 --> 00:01:15.920
So that's what I want to talk about today is I want to talk about how you need to stop asking AI to disagree with you, to recognize that it is often a yes man that will tell you, no matter what you're doing, it's a great idea.
00:01:16.239 --> 00:01:19.519
So, welcome to episode 145 of The Crazy One.
00:01:19.599 --> 00:01:21.120
As always, I'm your host, Stephen Gates.
00:01:21.280 --> 00:01:27.120
This is the show where we talk about creativity, leadership, design, and everything else that matters to creative people.
00:01:27.280 --> 00:01:28.640
Now, as always, appreciate it.
00:01:28.719 --> 00:01:34.480
Leave a review, tell a friend, hit the subscribe button so you can once again get the new shows whenever they're coming out.
00:01:34.719 --> 00:01:41.200
Because thanks to AI and my new writing partner, which is Claude, I've got the next 10 episodes already written.
00:01:41.439 --> 00:01:51.120
I finally have got a better handle on what for me have been the two most time-consuming parts of the show, which are writing the episodes and then handling all the content production afterwards.
00:01:51.280 --> 00:01:57.040
So I found a better way of doing it that's going to let me put out content on a much more regular basis.
00:01:57.519 --> 00:01:59.359
But but let's talk about this, right?
00:01:59.519 --> 00:02:12.560
Because I think that in so many ways, it's a trend that I started to notice probably about like two years ago, where and it's honestly one of the reasons why I left Chat GPT because it just was so annoying to me that it was just such a yes man.
00:02:13.039 --> 00:02:22.000
And this is the thing, is that I think right now you sort of see like AI adoption is everywhere, but I think so much of AI literacy is still in such a primitive state.
00:02:22.240 --> 00:02:28.879
Because I think, look, most people have learned it sort of the way they almost treat it like a search engine, right?
00:02:29.120 --> 00:02:32.560
Write this, summarize this, give me an idea.
00:02:32.800 --> 00:02:36.000
But they're prompting it for an output.
00:02:36.560 --> 00:02:37.840
And I get it, right?
00:02:37.919 --> 00:02:46.159
Like again, if you come from search and a lot of the other ways we've used technologies that sort of looked similar, that was this the way you would do it.
00:02:46.240 --> 00:02:52.400
And so I think that's why like this habit got baked in early, and there aren't that many people that are really questioning it.
00:02:52.719 --> 00:02:58.639
But I think that's the thing that I'm seeing is that AI has just become this like confirmation machine.
00:02:58.879 --> 00:03:04.240
It reflects back what you're already thinking, but somehow it's it's doing it faster and it sounds more confident.
00:03:04.960 --> 00:03:08.960
I was watching an episode of last week tonight, I don't know what, a week or two ago.
00:03:09.120 --> 00:03:17.360
There's a whole story on there on AI chatbots, and how I think it was Chat GBT had convinced a man that he'd come up with a totally new form of math.
00:03:17.520 --> 00:03:21.199
He'd discovered all this new sort of thing, and he would prompt it and say, like, are you sure?
00:03:21.360 --> 00:03:23.120
And it would say, Yes, like you're a genius.
00:03:23.280 --> 00:03:24.639
You've come up with this thing.
00:03:24.960 --> 00:03:28.719
Well, of course, in the end, none of that was actually real.
00:03:29.520 --> 00:03:31.120
But I think that's the thing, right?
00:03:31.199 --> 00:03:37.360
Is it it's that confidence with which it answers that's the dangerous part.
00:03:37.840 --> 00:03:50.319
Because again, I think it it says things like it's the authority, whether it's right or wrong, whether you understand the sources it's citing are good or bad, like whether it's like it's just it's giving you this false sense of confidence.
00:03:50.560 --> 00:03:55.439
And I think that's really the step before plausible noise, right?
00:03:55.520 --> 00:04:00.000
That's why people feel confident to do everybody else's job, to do these sort of things.
00:04:00.400 --> 00:04:02.400
And and that's the problem, right?
00:04:02.479 --> 00:04:04.560
Is because everybody's like, oh, it must be right.
00:04:04.639 --> 00:04:06.080
It told me that.
00:04:06.560 --> 00:04:08.240
But here's the thing that I've learned.
00:04:08.479 --> 00:04:18.480
If you aren't actively pushing back, if you aren't using it almost like a sparring partner, then you're getting almost your own assumptions dressed up in better language because it's a learning model.
00:04:18.560 --> 00:04:22.639
It learns what you want, what you like, and what you want.
00:04:23.680 --> 00:04:28.800
And I think that that's where for me, like I said, I try to use it almost like a sparring partner philosophy.
00:04:28.879 --> 00:04:30.319
This is how I've always treated it.
00:04:30.480 --> 00:04:41.600
And I think it really changes everything that you get out of it, and that you can start to get that force multiplier without losing all the best parts of what makes us all creative and special.
00:04:42.240 --> 00:04:49.279
Because I think look, at the end of the day, I think in a lot of cases right now, AI is just going to tell you what you want to hear, right?
00:04:49.439 --> 00:04:51.120
It's trained to be helpful.
00:04:51.279 --> 00:04:54.959
And I think in too many cases, helpful means it's trained to agree with you.
00:04:55.199 --> 00:04:58.000
So you, if you ask it like, hey, is this a good idea?
00:04:58.160 --> 00:05:01.439
It will find a reason to tell you why it is.
00:05:01.839 --> 00:05:11.600
Again, there was Ryan Sirhans, a big um real estate broker in New York, said he lost a deal because of Chat GPT, because the seller went in and said, Am I getting enough money?
00:05:11.680 --> 00:05:12.560
And it said, No, you're not.
00:05:12.720 --> 00:05:13.680
Here's the comps why.
00:05:13.759 --> 00:05:15.759
And the buyer went in and said, Am I paying too much?
00:05:15.920 --> 00:05:17.279
And it said, Yes, you absolutely are.
00:05:17.360 --> 00:05:18.720
Here are the comps why.
00:05:18.959 --> 00:05:24.480
So both sides, empowered by Chat GPT, were convinced that they were right and the deal blew up.
00:05:25.120 --> 00:05:28.319
Again, I think if you ask it, you know, is this a good idea?
00:05:28.560 --> 00:05:37.920
And I think if you ask it even how will this fail, it will find those reasons too, because like the model doesn't have conviction.
00:05:38.480 --> 00:05:39.040
You do.
00:05:39.120 --> 00:05:47.120
So I think that like the output quality isn't always necessarily the problem, but it's it's questioning that quality that is.
00:05:47.839 --> 00:05:58.319
And I think a lot of people have fallen into this bad habit where they're sort of use it to validate what they're already thinking, as opposed to like really using AI to find the weak spots.
00:05:58.399 --> 00:06:10.079
And I think that's always been my approach, which is why I fell into it with AI, is once I come up with an idea, I want to use research or data or testing or whatever to break it.
00:06:10.319 --> 00:06:11.600
I want to find the weak spots.
00:06:11.680 --> 00:06:19.040
I want to find how it can be better, as we're with too much research and too many other creatives, they're going out there to find validation, right?
00:06:19.120 --> 00:06:22.000
They're going out there for somebody to tell them it's a good idea.
00:06:22.240 --> 00:06:27.360
And now with AI, it's created a smaller echo chamber for that to happen.
00:06:27.759 --> 00:06:38.160
And I think, you know, for me, that's where it's like we need to start to think through like, how do you work with the smartest, most challenging person you know?
00:06:38.560 --> 00:06:40.160
We we all have worked with them, right?
00:06:40.240 --> 00:06:42.959
Like that person that's just like they always seem to have the answer.
00:06:43.040 --> 00:06:46.000
They can see it from another angle, they're always gonna challenge what you're thinking.
00:06:46.160 --> 00:06:53.120
The best leaders, the best CEOs, the best, like kind of again, real thinkers I've ever worked with, this is what they did, right?
00:06:53.199 --> 00:06:56.720
And and you wouldn't just present them with an idea.
00:06:57.040 --> 00:07:00.879
You'd go in and know that it would be a debate or an argument that they were gonna push back.
00:07:00.959 --> 00:07:06.160
You had to defend your position or revise or see where there were holes in it.
00:07:06.399 --> 00:07:12.720
And I think that AI can be that way, but only if you set it up to be that way, right?
00:07:12.879 --> 00:07:14.879
Because look, the tool doesn't change.
00:07:15.040 --> 00:07:18.319
I think that your posture towards it needs to.
00:07:18.879 --> 00:07:25.360
Because for me, any idea, anything where you've done the thinking always survives that fight, right?
00:07:25.439 --> 00:07:26.959
It survives the pushback.
00:07:27.279 --> 00:07:34.560
And in general, I think this is just a culture we need to build more of, where I've talked about we need to kill status meetings and start having decision logs.
00:07:34.800 --> 00:07:36.959
This is why this needs to happen.
00:07:37.199 --> 00:07:41.759
Because look, I think that like every strong idea will survive that fight.
00:07:42.079 --> 00:07:46.240
And AI, in a lot of cases, I think is is one of the easiest.
00:07:46.399 --> 00:07:53.040
I don't know if I'd say safest, I'm not sure that feels right, but it's one of those places to have that fight before the room does it for you.
00:07:53.199 --> 00:07:57.519
To start to see where are your blind spots, where are the weak points in sort of doing this.
00:07:57.759 --> 00:08:05.519
And I've developed what are really kind of four different types of what I'll call like challenge prompts.
00:08:06.000 --> 00:08:08.720
And and this is again where I'm going to go back.
00:08:08.879 --> 00:08:13.199
One, I always show up with an idea, but then I want to find the holes in it.
00:08:13.439 --> 00:08:16.879
So there are four different sort of ways that I've tried to do this.
00:08:17.199 --> 00:08:19.839
The first one is what I'll call like a failure prompt.
00:08:20.000 --> 00:08:24.639
And that's where I what I'm gonna do is I want it to make it argue with me, right?
00:08:24.879 --> 00:08:27.199
By asking it, like, look, how's this gonna fail?
00:08:27.279 --> 00:08:30.399
What would the most skeptical person in the room say?
00:08:30.639 --> 00:08:33.840
Because what I want to do is I want to force it off that happy path.
00:08:34.080 --> 00:08:37.120
I want to force it off that, like, yes, like Steve, you're brilliant.
00:08:37.200 --> 00:08:37.840
This is great.
00:08:37.919 --> 00:08:39.279
You're gonna go change the world.
00:08:39.440 --> 00:08:41.519
Because can't I just I don't need that shit, right?
00:08:41.600 --> 00:08:42.960
Like, I just that's not what I'm looking for.
00:08:43.039 --> 00:08:44.720
That's not gonna help me in my work.
00:08:44.960 --> 00:08:49.519
So again, I think that sort of thing of like, look, what's my biggest critic gonna say about this?
00:08:49.919 --> 00:08:52.720
Forces it down a different line of thinking.
00:08:52.960 --> 00:09:00.720
I think the next one for me is to really try to try to find prompts that will expose assumptions.
00:09:01.360 --> 00:09:06.639
Because look, I think that like, how do I surface and find what I'm taking for granted?
00:09:06.879 --> 00:09:10.559
So this is asking at things like, what does this idea assume is true?
00:09:10.799 --> 00:09:13.039
What would have to be true for this to work?
00:09:13.200 --> 00:09:13.440
Right.
00:09:13.519 --> 00:09:16.799
Because look, I think most ideas don't fail on the surface.
00:09:17.039 --> 00:09:30.720
They fail because of like a buried assumption that either everybody misses or that nobody checked, or that there is something in there where it's like, you know, this is gonna be great, but it's gonna die in legal or tech or like somewhere else, right?
00:09:30.799 --> 00:09:39.039
Like something that I'm going to assume that they can do it and that it all sounds very like magical and great to go pitch, but it's not gonna get out the door.
00:09:39.840 --> 00:09:51.759
I think that a lot of it also for me is that again, whenever it states things back with such confidence, the third thing I want to do is really sort of just have prompts that make it prove itself and really challenge its data validity.
00:09:52.080 --> 00:09:55.519
In asking things like, again, how confident are you in this and why?
00:09:55.679 --> 00:09:59.200
What's the strongest argument that, you know, in what you just told me?
00:09:59.360 --> 00:10:01.759
What are the actual data sources that you are citing?
00:10:01.840 --> 00:10:16.159
If you are building your own models or your own projects or doing those sort of things, to bake that into the instructions, that I only want you to be able to find like legitimately, like legitimate data sources that can be vault that can be verified from multiple different directions.
00:10:16.320 --> 00:10:25.039
And that again, I don't want you finding like just kind of one crackpot's blog who quoted a bunch of stuff and then just assuming that that's the right data.
00:10:25.279 --> 00:10:34.080
So I think this is also what I built into a lot of the models and a lot of the things that I work with, are where it's going to start to know that it needs to argue with me, not to agree with me.
00:10:34.159 --> 00:10:35.440
It's going to question those assumptions.
00:10:35.519 --> 00:10:38.960
It's going to show me what the data is that it's doing.
00:10:39.200 --> 00:10:49.120
And then the last thing is, and again, this is a concept that I've talked about in past shows, is I'm I want it to really almost start to run pre-mortem prompts.
00:10:49.360 --> 00:11:02.320
So this is like, right, like run the failure scenarios, run those sort of things that, you know, again, so it will say, like, look, this is a great idea, but like it's going to fail 12 months from now, and here's why.
00:11:02.639 --> 00:11:02.960
Right?
00:11:03.039 --> 00:11:09.360
Like, again, what's it'll start to, I'll start to ask it, like, look, what's the most expensive that we mistake that we could make here?
00:11:09.759 --> 00:11:12.320
And that's borrowed from product thinking, right?
00:11:12.399 --> 00:11:14.799
That's not just good practice in AI.
00:11:14.960 --> 00:11:19.279
That's actually just good product thinking and good systems thinking.
00:11:19.519 --> 00:11:23.600
But again, to start to run the pre-mortem prompts of like, what are the guardrails we should have?
00:11:23.759 --> 00:11:25.519
What are the like what's the biggest consequence?
00:11:25.679 --> 00:11:26.720
What's the blowback?
00:11:26.879 --> 00:11:29.200
Can we claw this back if this goes wrong?
00:11:29.360 --> 00:11:29.519
Right.
00:11:29.679 --> 00:11:30.720
There's those sort of things.
00:11:30.879 --> 00:11:35.600
And I think I've got an episode coming up that just focused on pre-mortems.
00:11:36.000 --> 00:11:38.559
I'm thinking now that probably should be the next episode.
00:11:38.799 --> 00:11:44.480
But for me, it's that like it's the failure prompt, the assumption exposure, data validity, and pre-mortems.
00:11:44.720 --> 00:11:50.320
Those are the sort of things that you need to be getting it to do, baking it into your models to push back.
00:11:50.559 --> 00:11:50.879
Right.
00:11:50.960 --> 00:11:55.600
And I think that in a lot of cases, like it's making this a habit, not just a checklist.
00:11:55.919 --> 00:11:58.639
Because this only works if it becomes part of the way you think, right?
00:11:58.720 --> 00:12:02.399
Not a step that you remember at the end, like, oh, did somebody ask legal?
00:12:02.639 --> 00:12:07.919
But before, right, like you take any AI output seriously, ask what it got wrong.
00:12:08.080 --> 00:12:11.440
Before you act on any AI-assisted idea, right?
00:12:11.600 --> 00:12:12.879
Run the pre-mortem.
00:12:13.039 --> 00:12:17.519
Before you present anything built with AI, stress test the assumptions.
00:12:17.840 --> 00:12:21.840
And again, my thing isn't that like it's to distrust the AI.
00:12:22.399 --> 00:12:30.799
It's it's in using it in a way that you would with any smart colleague, with any sparring partner, with any great partner you've ever had, right?
00:12:30.879 --> 00:12:32.960
They don't just take it at face value.
00:12:33.120 --> 00:12:38.159
They push you, they push your thinking, they don't just take the first answer as gospel and agree with you.
00:12:38.320 --> 00:12:44.000
The weakest organizations and the worst leaders, I've talked about on here for a freaking decade, right?
00:12:44.240 --> 00:12:47.440
Are the ones that surround themselves with people who disagree with them.
00:12:47.679 --> 00:12:49.679
It's that yes people culture, right?
00:12:49.759 --> 00:12:55.679
Like everybody on my team thinks I'm brilliant, but somehow like our ideas aren't going anywhere, our stock price isn't up, right?
00:12:55.759 --> 00:12:56.879
Like that's why.
00:12:57.600 --> 00:13:06.879
And look, and I think that here's the harder truth in all this is that I think the reason people don't do this isn't that they don't know how.
00:13:07.440 --> 00:13:09.120
It's that they don't want to.
00:13:09.759 --> 00:13:10.159
Right?
00:13:10.240 --> 00:13:12.559
Like confirmation feels good.
00:13:12.720 --> 00:13:14.960
Like getting validated feels comfortable.
00:13:15.120 --> 00:13:21.519
Saying, oh, look how smart I am and how well I'm using AI, and I'm a genius, and I'm out on the frontier of this, right?
00:13:22.399 --> 00:13:24.639
That feels better.
00:13:25.279 --> 00:13:29.120
But my thing is at the end of the day, that's a short-lived, short-sighted approach to these things.
00:13:29.200 --> 00:13:33.360
Because look, if your idea survives the challenge, then you can present it with real confidence.
00:13:33.519 --> 00:13:40.080
If you're somebody that isn't great at research and that's not what you like has always been your strong suit, then this is a great tool to be able to do this.
00:13:40.320 --> 00:13:41.440
Because that's the thing, right?
00:13:41.759 --> 00:13:48.480
If it doesn't survive the test, you just saved yourself walking into a room and getting destroyed, right?
00:13:48.639 --> 00:13:51.919
Of presenting something that is going to erode what you are doing.
00:13:52.000 --> 00:13:59.600
And that's why I said, like, so many people seem to be on this road to destruction, but they're so happy about it because they're feeling validated and they're doing it quickly.
00:13:59.840 --> 00:14:03.360
But if you're doing that, look, at the end of the day, either way you win.
00:14:03.600 --> 00:14:11.519
Because like if you're willing to have that fight first, if you're willing to test that, if you're willing to get into that habit of making your ideas stronger, your work just gets better.
00:14:12.080 --> 00:14:13.200
And that's the thing, right?
00:14:13.279 --> 00:14:21.600
The people who are going to be the most impactful, the most dangerous, the most the ones who sort of really excel, they're not going to be the ones that are using the best tools.
00:14:21.840 --> 00:14:28.399
They're going to be the ones that are asking the best questions, that are building the best models, that are finding the best workflows.
00:14:28.559 --> 00:14:33.279
And that's why I keep coming back to like this is AI is not a technology problem, right?
00:14:33.360 --> 00:14:41.840
It's a culture problem, whether it's the culture of a company, the culture of a team, or your individual kind of way of working in that kind of like microculture.
00:14:42.639 --> 00:14:43.919
Because that's the thing, right?
00:14:44.000 --> 00:14:52.879
Like take the idea you're working on, the one you're doing right now, or why the one that maybe you're in Claude or ChatGPT working on as you're listening to this show right now, today.
00:14:53.039 --> 00:14:57.200
And before you do anything else with it, run through these prompts, right?
00:14:57.360 --> 00:15:00.559
Do the failure, the assumption, the validity, and the pre-mortem.
00:15:00.799 --> 00:15:02.320
Those four rounds, right?
00:15:02.480 --> 00:15:05.120
It's going to take you 15 minutes tops.
00:15:05.600 --> 00:15:06.879
See what survives.
00:15:07.039 --> 00:15:08.080
See what doesn't.
00:15:08.240 --> 00:15:11.759
Because look, whatever's left after that fight is going to be worth your time.
00:15:11.919 --> 00:15:16.320
And that you being comfortable and validated is not going to get you where you need to go.
00:15:16.480 --> 00:15:17.519
I know it feels better, right?
00:15:17.600 --> 00:15:18.879
Like, oh, you're great.
00:15:18.960 --> 00:15:29.120
And and like, look, I mean, you can tell that there are certain engineers of a certain generation that just program things in a certain way, focusing on that validation.
00:15:29.600 --> 00:15:30.559
But that's the thing, right?
00:15:30.720 --> 00:15:33.600
AI is not going to make you smarter by default.
00:15:33.919 --> 00:15:34.240
Right?
00:15:34.320 --> 00:15:39.519
It's not going to make you faster if like than a level of thinking that you're already doing.
00:15:39.840 --> 00:15:40.159
Right?
00:15:40.320 --> 00:15:45.759
You have to decide to think harder, to be able to hold yourself to a better standard.
00:15:45.840 --> 00:15:47.120
Then you use the tools.
00:15:47.279 --> 00:15:49.120
Then you go in and do this.
00:15:49.360 --> 00:15:51.279
That's how, again, you want to stay ahead of it.
00:15:51.360 --> 00:15:52.879
That's how you don't become commoditized.
00:15:52.960 --> 00:16:00.879
That's how we talked about in the like the last episode, where you're starting to move up into those top two quadrants where you are more of a strategist.
00:16:00.960 --> 00:16:01.919
You are more of a thinker.
00:16:02.000 --> 00:16:06.559
You are doing more of those sort of things where, again, it's not just taking it at face value.
00:16:06.639 --> 00:16:08.159
Because that's where I see most people screw up.
00:16:08.240 --> 00:16:14.080
It's this copy and paste version or like Google search version of using AI.
00:16:14.399 --> 00:16:17.120
And that again, think of it like a sparring partner.
00:16:17.279 --> 00:16:21.200
Think of it like that best partner, the person that challenged you the most.
00:16:21.360 --> 00:16:26.960
Because if you can get it to do that, then you can really start to have a big unlock in the way that you're working.
00:16:27.039 --> 00:16:28.960
That's going to be a big help.
00:16:29.200 --> 00:16:30.480
But I'm also curious, right?
00:16:30.559 --> 00:16:41.120
Like as always, if there are ways that you're working with it, if there are tricks that you've found, if there's other techniques that you found is sort of pushing back, if there's instruction sets that you've done or models that you built, or public ones that you want to share, let me know.
00:16:41.200 --> 00:16:42.799
I'm happy to share them out to people.
00:16:42.960 --> 00:16:44.000
But that's the thing, right?
00:16:44.080 --> 00:16:48.720
Is that we have to start normalizing this sort of thing to be able to get things to be better.
00:16:49.039 --> 00:16:51.679
So as always, thanks for your time.
00:16:51.840 --> 00:16:52.799
Thanks for listening.
00:16:53.039 --> 00:16:53.919
I hope this helps.
00:16:54.080 --> 00:16:58.879
I hope that it helps you think through your work a little bit differently and try those things a little bit differently.
00:16:59.039 --> 00:17:02.399
And hey, as always, stay crazy.
