ABOUT THIS EPISODE
Professor Paul Cairney reminds us that when we produce solutions to complex problems, our biggest problem still lies ahead: getting people to notice. We should adjust our expectations accordingly, and work patiently.
David Walker: Before we start this episode of Shorewalker On Reports, a brief note. We spend a lot of time today talking about government policymaking. But many of the ideas in this episode also apply well beyond the world of government.
And the piece of this week’s episode that I want to play first is an example of this. Here’s our guest, political scientist Professor Paul Cairney, tackling a tough issue head-on: typically, when you publish a report, you don't get listened to. You're transmitting, he says, but your audience is usually, typically, not receiving.
Paul Cairney: So I would say that most conversations are not meaningful, and most intended exchanges of information are ineffective, or they don't happen. And what I usually mean by that, is that I think most people, when they convey information and reports, are just sending that information out into the world. They're signalling as if you could connect your message in a straight line to your audience – you know, direct transmission of information. And I think increasingly, people are much more cognisant of the idea that people will only remember and act on, or really pay attention to, the information you give them if it's part of a meaningful exchange of information, or you have some kind of non-trivial relationship with that person – they know you, or they have some other reason to think of you as an authority, someone to be listened to, and to invest the time [in]. I think otherwise, you're just sort of sending information out into a vacuum.
David Walker: So when do we just send information out into a vacuum? And how can we achieve more than that? Those are the big questions of this podcast. And Professor Cairney has answers – in fact, some of the best answers I’ve heard anywhere. Indeed, Professor Cairney is the source of what I have dubbed Cairney’s Law of Reports.
But before we get to Cairney’s Law, who is Paul Cairney? As you may just have deduced from his accent, he’s a Scottish academic. He works at the University of Stirling. But he also works with UK policymakers and has actually written reports for government.
Paul Cairney: I primarily conduct and teach research, largely at postgraduate level. And I suppose my way into this kind of field was through the study of policymaking, which is useful, but also picking up on the concerns of people outside of policy studies about the weak connection between their research and policy. So there seems to be a space there to translate policy studies insights into how people could understand the use of evidence and policymaking and adapt to suit. I've also, you know, throughout that time, have been involved in regular sort of academic practitioner exchanges, either giving talks or being part of workshops, and I've done things like short courses on civil service training, and I have written the occasional report for audiences and policymaking.
David Walker: You've worked with a bunch of practitioners in the policy field, and you've done this work yourself. Can you give me a very high-level overview of what you think people get right and wrong when they're creating public reports?
Paul Cairney: I don't want to do researchers down too much. But you know, the kind of caricature of a researcher is: they ask their own research question; they produce their own research, and then at the end, they think about who else would be interested in this and they do some recommendations.
Whereas I think the art to writing policy reports is, you start from the other direction. Who is either my client or my audience? What are their beliefs or interests? What will catch their attention? What are their expectations? And how can I tailor what I do to fit with what they are doing? And I think it's just asking for a simple shift in mindset towards what people think they are doing. And a lot of that is about giving up control. You know, I think researchers particularly in my field … like to be in control of the research, whereas this is about really giving up that control and trying to serve some different purpose.
Then I think the usual other advice is about things like plain English, or plain speaking, brevity. Don't write 1000 pages – or if you're going to write 1000 pages, at least provide one page that they’ll actually read. And focus on what people have the capacity to do rather than what you would like them to do. And I think the classic trade-off there is between wanting to look comprehensive – you know, I have covered everything, and I knew all this stuff, and here it all is – and what you can make memorable, which is the opposite of comprehensive – it's the smallest number of things that you can provide them to give them a proper sense of what's going on. Because you know that they'll only pay attention to the small amount, or they'll only remember a small amount of what you've told them.
David Walker: That is a very strong strand in the work of yours that I've read. And it's a really quite striking theme, partly because it's so uncommon. To be telling people that most people are not very interested in most of what you have to say is something people seem reluctant to do.
Paul Cairney: I say it with a smile on my face, usually. I don't think it should be such a hard sell if people just reflect on how they process information or what they pay attention to. Because I think the basic point is, there's an almost infinite or overwhelming amount of information out there. And we only have a finite ability to pay attention to it and process the information. So of course, we're making all sorts of choices. And we're having to ignore almost everything almost all the time to be able to function, to make choices. So I think it's probably a just a hard sell. Because if you do research or you're looking to provide advice, the inclination, or the incentive, quite rightly, is to produce a comprehensive, thorough report, because policy problems and policymaking is complex and hard to understand. But at the same time, people only have this ability to understand small parts of it. They will interpret it in different ways. So it shouldn't be such a hard sell, I think. It shouldn't be so unusual.
David Walker: Hearing Paul Cairney say this – and hearing him say it this bluntly – was a bit of a revelatory moment for me. “We're having to ignore almost everything almost all the time.” I've decided this really deserves to be called Cairney’s Law.
Now, Cairney argues that Cairney’s Law has been around a long time. He thinks of it as a variant of the economist Herbert Simon’s 1957 concept of “bounded rationality”. But I think he’s being modest.
Paul Cairney: I think that's just me translating the most common insight in policy studies. So in my fields, usually, the phrase we use is “bounded rationality”. And the comparator is this idea of comprehensive rationality, which is we ascribe in people, including policymakers, the ability to process all information, you know, make consistent choices, rank them in order of preference and know exactly what happened when you make choices. And so instead, I think that the person most associated with that, or who came up with that phrase, is Herbert Simon, who talked about what actually happens when you ascribe to people, human abilities or organisational abilities where they can only process a small amount of information. And I think that's just people in my field for decades have just tried to use that basic insight and ask, “what do people actually do with that problem?” You know, how do they combine cognition and emotion to make choices quickly? And how do they decide what or who to pay attention to and who to ignore?
David Walker: These are some of the toughest and most important questions we face. How can decision-makers make choices quickly? How do they choose what to pay attention to? Cairney says that in this time-pressured environment, with limited information, policymakers usually do not take up new ideas straight away.
So policymakers don’t immediately select most of the ideas they’re offered. And over time, Cairney says, most of the people who make a case to policymakers for a particular course of action come to accept this. There’s a gap between what you’d like to happen, and what will actually happen right now.
Indeed, you might call this Jagger’s Law: you can’t always get what you want.
Paul Cairney: I do regularly speak with people who work in government, or with government. And now, this isn't a uniform experience, but it's typical: you have people who have certain expectations before they go in, and they’re high expectations based on what they require from policymaking. You know, they need things to change. Because there are a huge number of policy problems that exhibit crises and inequalities and such-like. So they need things to happen, and they have evidence, or they think they have evidence, on how to solve it. So they go into government or with government thinking “Okay, this process should work this way to get what I want”. But then, whenever I've given a talk or spoken with people who have experience – we actually use the same language – they either talk about this gap between their expectations and what happens, or they talk in a different language about how you really have to come up with ways of working to be pragmatic within these systems, knowing that you can’t just get what you want, that you're going to be in it for the long haul, and [that]people might pay attention to your evidence at some time, but you're not really in control of that. So I think if people are experienced in processes, they come up with a different way to say that thing, and they would give similar advice. I think the only people who don't think that way are people who are … I wouldn't quite say idealists, but they have certain expectations about policymaking that is not tethered to experience or reality.
David Walker: So why is it that people so often can’t get what they want out of policymakers? Why is there such a gap between what the people who contribute to reports want to happen, and what policymakers end up doing? Why do the people who write reports so often feel that they remain unheard?
Paul Cairney: It's funny. So I gave a talk recently, and the feedback was, you know, I was describing a dysfunctional system or a chaotic system or something like that. This is the impression that some people have of policymaking – that, you know, no one's in control, that sort of thing. Or you talk about the incompetence of politicians, or that they're bad actors, or something like that. And I think what I try and do is separate out two things. One is, what are you experiencing that relates to specific individuals who are not acting in a proper manner – you know, according to your expectations? And what are the things that you have to expect, regardless of the person you're speaking to or in charge? You know, so I would say some of them are individual, but a lot of them are systemic, you know. And so it's the systemic type of factors that I talk about. And I think the most useful, or at least the most frequent, engagement I have with that topic, is I'm often asked to give this talk about, the question is along the lines of: 'why do people ignore my evidence?' Or: 'why do they ignore the evidence, or the best evidence?'
And I think just going through very briefly, three answers to that question would give my sort of position on that sort of systemic thing.
Because the first answer is that we don't actually agree on what the evidence is, or what is the best evidence and how to gather it. You have different people in different fields, who often have very fixed ideas about what is good evidence in relation to the methods they use. But those ideas are not shared across the profession. And they're certainly not shared within policymaking. So I think the classic example is some people think that the gold standard is the randomised control trial [RCT] or the systematic review of those trials. Some people in policymaking either have not heard of RCTs or they have no real experience of them and they don't give them any priority at all. In fact, they would often give much more attention to qualitative evidence or, you know, basic statistics in a spreadsheet ,or that sort of thing. So there's just this basic lack of agreement on what the evidence is. And I think that once you accept that, then the question doesn't make sense anymore. Why did people ignore the evidence? Well, if you can't establish what the evidence is, or you don't recognise that the evidence is contested, the question doesn't make any sense.
The second answer is what we've talked about already, which is, you know, just given the context they face, they have to ignore almost everything almost all the time. And so they're trying to find ways to prioritise some information at the expense of the rest. And I think the best advice is to try and work out how people think so that you can work out how they prioritise information, and fit in.
But the third one is the thing I study most, which is the policymakers are not in control of the policy systems in which they operate. So I think the classic reference point, particularly in Westminster systems, is this idea of a really simple policy cycle where there are clear stages, and you can anticipate the use of evidence. So you know, first produce evidence to determine the size and urgency and scale of the problem, then produce evidence to come up with solutions that will work, then choose the best one, and then use evidence to evaluate. Now I think if the policymaking worked like that, you could be disappointed at the lack of connection between research evidence and policy, because there's a clear process that everyone's involved in. But I think as soon as you give up on that idea, and you accept that policymaking is spread much more over many different venues, and there's no simple process that connects everyone, then I think you give up on the idea that there ever could be this direct connection between the evidence you present in a report, and your audience doing something about it.
David Walker: So here’s where Paul Cairney has taken us so far. Policymakers have to ignore almost everything almost all the time, just to be able to make decisions quickly – that’s Cairney’s Law. And so when you’ve written a report, policymakers won’t usually give you the decisions you want – or at least not right away. That’s Jagger’s Law: you can’t always get what you want. And so most people end up thinking that the policymaking process is dysfunctional, and maybe that politicians are incompetent.
And Cairney’s response to all of this is that you might want to adjust your expectations. The evidence for a particular course is highly contested, and policymakers can’t weigh up all of the evidence with perfect rationality, and there’s rarely any neat, straight-line process that produces the ideal result.
Cairney bases this thinking on the work of a political scientist named John Kingdon, who has – full disclosure – heavily influenced my own thinking about policymaking. Kingdon is famous for an approach called the ‘multiple streams approach’. And Cairney often emphasises Kingdon’s work. I’ll link in the show notes to a paper he co-wrote on Kingdon’s approach - Paul Cairney and Nikolaos Zahariadis, Multiple streams approach: a flexible metaphor presents an opportunity to operationalize agenda setting processes, 2016.
In that paper with Nikolaos Zahariadis, Cairney argues that policy contributors should remember this:
- In the policy process, attention is scarce. Remember Cairney’s Law
- Many different elements and voices influence decisions.
- It’s fluid and often non-rational.
- Luck plays a bigger role than you might think.
So what should we do with all this? How should we react to these policymaking constraints?
Cairney argues that policy contributors should prepare themselves for those moments when a window of policy opportunity opens. People need to show patience; those moments can take years to arrive. But when some new problem or crisis does appear, smart contributors to policy should have solutions to policy problems and crises worked out and ready to go. Often they should already have those ideas in circulation in the policy world. Here Cairney talks about Kingdon’s work.
Paul Cairney: So I reckon when I look at studies of the use of policy studies and other fields – so say health, public health, that sort of thing – [then] multiple streams and Kingdon, I think is by far the most popular, I think because it looks like it gives you this hopeful message: if you if you can act like a policy entrepreneur, if you can invest your time to maximise the impact of your solution, you can make a disproportionate difference. And that is about working out how to exploit a window of opportunity. People will pay disproportionately high attention to an issue for a short amount of time. So you have to have the solution available that you can attach to a problem, and hope that they have their motive and opportunity to act and that can all happen in this notional window of opportunity. Now I think the tricky thing is how to convey how that relates to policy studies rather than individual psychology. Because I reckon when people think of a window of opportunity, they think of something akin to an individual exchange …
Now, I think what Kingdon is really talking about is this window of opportunity in a really complex, competitive, huge political system in which you are waiting for, you know, the correct conditions to arise. But he talks about [how] the amount of time you might have to wait can be anything from tomorrow to 10 or 20 years. I think there are those two things. Timing, of course, is important – but that timing is only to some extent in your control. His other metaphor is that these entrepreneurs, [you should] think of them as surfers waiting for the big wave, rather than being in control of the water or the sea.
So I think there is a good double message there, one is about trying to understand the political system enough that you can act effectively in it. But the other is to accept that you may be waiting a remarkably long amount of time for this thing to arise. It's not going to be … you know, you'd be lucky if it's tomorrow, and it might be that you're talking about [waiting] a career for the kind of change you're looking for.
David Walker: I confess I have my own favorite example of waiting a long time for a policy opportunity. It comes from the 2008 global financial crisis, which is still widely regarded as a high point for Australian economic policymakers. The Australian Treasury is the national government’s economic policymaking body, and it came to the crisis still remembering a bad policy call from 15 years earlier. In the early 1990s it had delivered a lot of stimulus in reaction to Australia’s 1990-91 recession. And not that unusually, they had delivered it slowly, so that much of it arrived in 1993 or even later, when the economy was already booming again. What happened next, according to a version I heard during the crisis, was that Treasury took this failure to heart. Treasury decided it needed to do better in the next economic downturn. And a senior economic bureaucrat named David Tune sat down with a few colleagues and wrote a report about what to do in the next big economic slowdown. According to the version I heard, that report lived in a bottom drawer in David Tune’s desk, literally or figuratively, for well over a decade. And then in September 2008, when the global financial crisis hit, they pulled it out. It had a pretty simple set of recommendations – recommendations that got actually transformed into a single phrase. The 1990s experience had taught Treasury that at the start of a slowdown, the government needed to use fiscal policy to quickly pump a lot of money into the economy’s household sector. They distilled that into a six-word imperative: “go hard, go early, go households”. They took that message straight to the Treasurer and the Prime Minister: “go hard, go early, go households”. And that message, delivered with real conviction, actually lodged in the mind of the people who would have to execute it really quickly – politicians who weren't career economics experts. It was adopted, and policy transformed within just a few weeks. That sort of speed is hard to get from, government, but they got it.
And Australia, almost uniquely among developed nations, didn’t go into recession in that period. So while I’m cautious about declaring government policies successful, it may be that this was indeed a success. And if it was a success, it succeeded because they Treasury had it ready when the need arose, and because Treasury communicated it clearly: “Go hard, go early, go households”.
David Walker: We shouldn’t finish this podcast without a closer look at Paul Cairney’s public policy textbooks and his accompanying blog posts. They’re not merely interesting. They also put some of Cairney’s ideas into action by presenting his ideas in multiple forms. You can buy the full version in print, and listeners to this podcast may well want to. But Cairney’s website also has pages explaining his ideas at less length for a different types of reader. Links to both Cairney’s books and to his blog are on this page.
Paul Cairney: The thing that got most attention outside of students and colleagues is the book called The Politics of Evidence-Based Policymaking. That wasn't a textbook; that was just a short book I wrote in 2016, and it prompted all these talks. The more recent book I've written is called The Politics of Policy Analysis. The difference with that one is, I wrote that as a series of blog posts first, and then I wrote the book. That was the first time I've done it like that. What I thought is that two-audiences idea: one audience will just read the website with the posts and a few podcasts; another audience will read the book. So that's probably the best example of a particular format. And similarly, the book that I've written that I'm most pleased with, that I'm actually proud of – because you know, you’re always a bit a little bit dissatisfied with what you've written – but the one I'm happiest with is called Understanding Public Policy in its second edition. And again, both editions come with a series of 1000-word or 500-word blog posts that explain the same stuff in a shorter and more concise way. So again, you've got a choice: do you want to read it in 500 words to get the idea, or do you want to follow up with a chapter, and a few podcasts. So I would say, if people are looking for a general introduction to the kind of policy concepts of talking about, like bounded rationality or complex systems, then it would be the 1000-word blog posts that are attached to the book that's called Understanding Public Policy.
David Walker: Paul, thank you for that. Reading the website and then talking to you has been tremendously interesting. I'm a newcomer to a field that you've obviously been operating in for a long time, and I'm sure there are other practitioners out there who are very good at what they do in this field too. But in terms of clearly transmitting the ideas – something which I appreciate more and more as I go on in life – you're very good, and it's been an absolute pleasure talking to you.
Paul Cairney: Thank you.
