OM DENNE EPISODE
The unfolding coronavirus pandemic is a story driven by numbers. But how reliable are the numbers we have? What can the data really tell us – and what are the major areas of uncertainty?
This is the transcript of a special episode of Risky Talk, recorded on 1st April. Michael Blastland and Professor David Spiegelhalter help us separate the signal from the noise.
***
Michael
Hello, and welcome to a special edition of a Risky Talk with Professor David Spiegelhalter. Except that many of you will know, I'm not David Spiegelhalter, and not because he's isolated with Covid 19. No, David is here or rather, at one end of a line with me, Michael Blastland, at the other. Plenty of distance between us. David, welcome to your own show!
David
Grand to be here.
Michael
We're doing it like this this week simply because the team who made the podcast decided we wanted to hear more from David on Risky Talk, where he's usually asking the questions. This time, he'll be answering them. And what we want to hear about especially at the moment, guess what, given that David's a world leading statistician, are his thoughts about coronavirus or covid 19: Where we are now, what the data we have can tell us and what it can't and how we should think about the risks. Now, we're having this conversation based on data available up to March 31st. So do bear that in mind if you're listening far into the future - new evidence is coming in all the time.
David, a more loaded question these days than it usually is. How are you?
David
Well, I'm actually feeling very fit because this confinement is making me pay a bit more attention to my exercise!
Michael
Do you have a sense of your personal risk? How risky does it feel for you?
David
Oh, yeah, not too bad. I wouldn't actually be too worried about getting it - I think it could be very nasty - but we'll come back later to how we can put our personal risks into perspective.
Michael
Alright, let's begin with how we should approach the problem because there are a lot of numbers kicking around. We've got numbers of deaths, we've got death rates, we've got numbers of cases, hospitalization rates, doubling periods, all of that. Which data would you focus on?
David
I think this is the first time in an international crisis like this, where there's been immediate access to huge amounts of data. But you know, the data that's most easily available is on cases and on deaths. And there's a problem with both of them.
It's one of these very basic things that, before getting down and doing any fancy analysis, we should ask: what are we counting in the first place? And with confirmed cases - those who've actually had a test and been positive - we know that's a very limited view of the actual number of cases out there. People say maybe there's 10 times as many. But of course, it depends on the testing regime.
Some countries, like Germany, have done a huge amount of testing and so the confirmed cases there might get closer to the number of people that have actually got it. But other places, like the UK - we've only been testing people in hospital, not even NHS workers really yet, and so actually the number of people that are in this country with the virus is a deeply unknown quantity.
With a common testing regime - as long as that's constant - you can use the confirmed cases to monitor what's going on, the trends up and down - provide some information. If the testing regime changes, of course, that all changes as well.
I guess you would think what would be a good measure, to put it very crudely, is counting the bodies. But we found out increasingly what a tenuous measure that is of the fatal harm of Covid 19.
Each country may be different in what it counts as a death. Some countries are counting deaths as where you can actually say ‘this is what caused the death’. In the UK, we were only just getting clarity from the Department of Health - they made this public just today - of what these deaths actually mean: that the deaths being daily reported are in fact, just deaths in hospital that have occurred for people who have tested positive. So, strictly speaking, if someone tests positive in hospital but actually dies of something else, they'll still be recorded as a Covid 19 death.
But those figures don't include any deaths in care homes outside of hospitals. And also the deaths figures are delayed. They're not the deaths that happened today or yesterday - some of them are coming in from two weeks ago. Hospitals have to wait until the family have been informed and given permission. And it seems clear also that they save them up over the weekend and send them all in on Monday. So these numbers are very erratic from day to day.
Michael
So we're seeing a bit of a spike on Mondays are we?
David
Oh, yeah, the Monday spike. In fact on Saturday and Sunday, the figures went right down. So we thought this can't be true: this is an artifact. And then suddenly they spiked - in a day they went from 180 to 380. This is completely impossible from a standard statistical perspective. Assuming there’s some sort of smooth process going on - we know what the variability should be - and what we see is vastly more than that. So we've got to be very careful in drawing conclusions about what's going on underneath.
Michael
Now given all these uncertainties, and they sound pretty large on all the elementary bits of data that we'd expect to have, can you draw any conclusions about the seriousness of the problem?
David
Both cases and deaths in the early part of the epidemic in almost every country increase by 30% per day. So you get a feeling that at least the rate of increase is representing what's going on, even if the absolute numbers are undercounted or whatever. From the rate of increase we can really get a feeling of how fast something is spreading throughout the population. Of course, the epidemiologists have built mathematical models that are being used to make predictions, and they have been using this kind of data to better inform the models.
Michael
So, I mean, I've seen one statistician, John Ioannidis, call it a “data fiasco”. Despite that, are there things we can say with some confidence?
David
Yes, by looking at the rate of change. We can get a lot of information from rates of change. Rates of change, are unaffected by systematic underreporting. They're unaffected by whether it's done per million or just in the whole country, and so on. So all these different ways of measuring make no difference when we monitor the rate of change. And that is the thing everyone is focusing on, because we're so interested in whether that rate is declining. In other words, what we'd love to see both in cases and in deaths is that this rapid increase smooths off into plateaus, into a constant rate, and then falls down as it did in China. And everyone's trying to work out what is going to happen in the Western countries.
Michael
So that's one good solid bit of evidence. What about the one that I think a lot of people are concerned with - the absolute rate of fatality. I've seen numbers around 0.1% or 0.2%. The Imperial estimate, I believe, is about 0.9%. Is there any way of adjudicating between them?
David
It's very difficult. It's highly contested. This is the infection mortality rate. The case fatality rate, which is how many people who we know got the virus actually end up dying, that can be estimated a bit more firmly. But the infection mortality rate, which is the percentage of people who get the infection and then die, depends crucially on of course, how many people have the infection - and that's the one thing we do not know! We have to just guess it until we have some population measures of who's had it. So eventually we will know (rather late…).
The Imperial group are working on 0.9% as an average, they released some analysis of China a couple of days ago where they lowered that to about 0.6%.
But other sources are saying much lower than that - 0.1% and so on - which is more like seasonal flu. Those studies are influenced by places like Iceland, who measured, you know, a lot of people, did a big lot of testing, - they managed to test around testing 3% of the population - and found a very small number of deaths. But that's Iceland! We're assuming that this is some constant number that applies just as much in Iceland, as it would in India or somewhere like that. And that is so deeply unlikely because whether you die or not, depends crucially on the health services available.
It will also depend on how serious the illness is that people get. For example, their viral load, how much have they been exposed to? We're getting multiple deaths of healthcare workers and it's really appallingly tragic, and that is sometimes in young healthy people, presumably because they've had a high exposure to the virus. And, in places where people live more distantly, less crowded in, might get very low exposure to the virus, and so actually get a much less severe form of the disease. So I think the idea that there is a fixed infection fatality rate, is pretty inappropriate. There will be big heterogeneity from place to place.
Michael
So would you feel confident about making a general comparison with things like normal flu or SARS or the 1918 influenza - to say simply ‘is this better or worse?’
David
Well, the standard thing that people are saying about it, compared to normal flu, is that it's twice as infectious and ten times as dangerous. And I think there is still reasonably strong evidence for that. It's different to SARS - SARS was far more lethal but actually much less infectious. And so we were actually able to keep it clamped down and controlled. I'm not sure about Spanish flu, I can't remember the numbers for that. But the crucial thing about that, actually, is that it hugely affected younger people. And that is not the case with this disease.
Michael
When we look at the various efforts by governments around the world to deal with this, there are a lot of lockdowns, but not everybody's locking down, and we're not all locking down in the same way. Can we work out which policies are making a difference yet?
David
Everyone's interested in this. And I don't want to make any grand claims at all. The plots I find most useful- and I'm looking at Our World in Data at the moment because I like their presentations a lot - is one of a log scale of cumulative deaths by country. And there, the gradient gives you a very powerful idea of the increasing number of deaths. Lines that run in parallel, are running at similar rates, the epidemic is taking a similar form. And we see for example, by that the United Kingdom at the moment is running very similar to Italy was at the same stage. People have been observing that Germany has got a very low mortality rate, but actually, although it's lower at the moment, its track is exactly the same as the UK's.
The very interesting controlled experiment in a sense is Sweden vs Norway. They're next door to each other and they've got very similar population structure, very similar climate, but they've taken extremely different policy views. Norway has implemented quite stringent criteria but Sweden, is still really quite relaxed, people going out for meals and no lockdown at all, schools still open and so on. Sweden's progress, although it's lower at the moment, seems to be following the trail of everybody else, of United Kingdom in Germany. Whereas Norway, actually - although it's still really barely started in Norway - is tracking quite low at the moment.
And of course, we've got Belarus who have volunteered to be the ultimate control in that they're taking no notice of the virus whatsoever.
Michael
I assume it will take a little while before we get a clear idea of how those policies have panned out?
David
Yes, exactly.
Michael
Okay. Well, we've talked about some of the limitations of these numbers. But let's add a few more complications (as if there weren't enough) and some more uncertainties. So, there's a lot of discussion about whether we're not really preventing many deaths, we're just delaying them and maybe not by very much, maybe because a lot of people are going to die quite soon anyway, if you're already elderly or very ill. What's your view about that?
David
Yeah, it's an interesting argument, and it's recognized by epidemiologists, they call it "mortality displacement". And it's often said that hot weather will do this. For someone who's frail and vulnerable, hot weather might bring their death forward a little bit. That's known as mortality displacement, or I've got to admit it, the technical term also used is ‘harvesting’!
Michael
Oh, gosh!
David
Yeah, exactly! I mean, I suppose it brings up images of the grim reaper, but that is the technical term. So people have said - in fact, Neil Ferguson, the head of the Imperial University modeling team, said in evidence to the parliamentary committee last week that he thought maybe two thirds of deaths were people who would probably have died reasonably soon anyway! So, there's a couple of issues about that. First of all it means, perhaps, that when we come out with this, we'll find that the number of excess deaths in the country over the year might very well not be excessive. It might look just look like even an average flu season in terms of the extra deaths.
Michael
Really!? Within the kind of normal range that we might expect to see?
David
Well it does vary very much - and that's to do with the flu. An average flu season will bring it up 17,000 or so extra deaths. A bad flu season would take it up to 25,000 extra deaths over the winter. Many of whom are, to use the technical term, ‘harvested’ from people who would die soon after anyway.
Michael
And that is assuming lockdown, in saying that it's within what we might expect?
David
Oh, yes, this is assuming lockdown! This is the crucial thing, this is what I dread at the end of this. If these lockdown measures are effective, and the aim is, as has been made explicit, is to try to keep the deaths below 20,000. If that happens, and people may look back at the end of the year and say, "oh, well, not many extra people died anyway", then the accusation will be made that therefore, we didn't need to do the lockdown, which is not a logical consequence! It's only as low as 20,000 because we have taken these measures.
The Imperial group estimated that, even with the less stringent mitigation measures which were in place before the lockdown - there would still have been 200,000 deaths. Now that really would make a dent in the population! We have around 600,000 deaths a year in this country, so we'll use that as the norm. So actually, you know, 20,000 extra deaths is within the margins what can happen in a flu year.
Michael
But even if we took even if we took two thirds off that 200,000 figure, we’re going to see something that's outside the normal kind of variation?
David
Yeah, and the crucial thing - and people do forget this, even though it's been emphasized again and again, by the Imperial modellers and everybody - is that in the end it’s not a matter of saving the lives of some old vulnerable people who are going to die soon anyway. It's the fact that if you didn't do that, if you didn't bring in these measures, you know, the NHS would be just totally overwhelmed, particularly as it’s short staffed. It's going to be touch and go, I think, anyway, whether it can deal with it.
I mean, amazing measures being taken at the moment and just imagine if this didn't happen, because, you know, for all these deaths, you've got 10 people maybe having to be admitted to hospital. Imagine if the health service was totally swamped by this, so that it just fell apart. It just fell apart. Imagine living in a society like that. What we'd feel about that.
So it's not a matter of just counting the bodies and saying, oh, wow, this is normal. It's a matter of, I think, keeping our respect in society.
There are some commentators who have said, Oh, well, this isn't cost effective. You know, if we just let all these old people die, we'd be saving all these billions of pounds. And we're spending too much to save these lives. And I think there are two answers against that. First of all, if you didn't take these measures, it would be vastly more. It'd be vast numbers. But even if you allow for that, that even those numbers would be very expensive - if you don't take account of this cost effectiveness analysis, which I'm a big fan, that sort of thing that NICE does in deciding about whether to pay for NHS treatments or not, it values a human life a year of human good quality human life out 20 to 30,000 pounds a year. Just like the Department of Transport values a human life at about 1.6 million pounds, and that's what they'll spend on road improvements that's expected to save one life. So the value money is put on the human life, and maybe in the end we'll have spent more than that to say these lives, but the road improvements and the NICE decisions are all to do with actually providing benefits to people. You invest in that and that will give someone benefit. What this investment is, is to stop people being enormously harmed.
Michael
There is an old argument that the great benefit, or one of the great benefits, the NHS brought was to lift the sense of fear. That people no longer had to wonder whether they had the resources of their own in order to make sure they were going to live. And that fear is one of those very general dispersed emotional qualities which we'd all probably pay quite a lot to get rid of.
David
Yeah, and just think how fearful you would be if you thought if you're going to get ill, there's no ambulance that would come, there was no bed available, there was no treatment available for you. You just had to lie at home and suffer and possibly die unnecessarily. And that's what would happen in this country if the thing was allowed to really let rip.
Michael
So the articles, I saw one this morning, saying that lockdown wasn't worth it for this reason, - you know, where we're probably not saving that many years of additional life and we're doing so at a cost of lives lost through deep recession that might outweigh the gain. You're not sympathetic?
David
No, I'm not sympathetic at all. But the one thing in that article I was sympathetic about is to say that this is not a matter of lives versus money. This is lives versus lives. We really do have to think about the fact that this intervention is itself causing great harm to individuals and society, both in terms of mental health, domestic abuse, unemployment, poverty, and so on. But as people have written as well, it's a bit like after the financial crisis: the financial crisis didn't hurt anybody. What hurt people was the steps taken after it in order to return to normal: the austerity. People can argue about what the impact was, but it was huge in terms of public health. I believe that and similarly for this: the crucial thing about this is about the harms done after this. Obviously there are very short term harms in terms of employment and so on, but the long term harm is not determined by the epidemic, it is determined by what is done after the epidemic.
Michael
In the end, how will we know how dangerous the virus has been overall? Do we have to look at 'all cause mortality'. Something really big, in the round, you know, some great aggregate calculation?
David
It's going to be very difficult. And I think we will be looking at excess deaths at the end of this. And as I said, I think at the end of it the excess deaths may not be that huge. We'll have various numbers, we'll have the number of death certificates on which it has been written. And that data is coming in now - very valuable. And we'll have the number of cases that have been reported from hospitals, we will have these sources. But in the end I would predict that it's not going to make a big dent in the overall mortality of the country.
Michael
Thinking about this, as you said right at the beginning, in terms of your own sense of personal risk: let's assume the numbers we have are ball park okay - can you put them in the context of all the risks we run in normal life? And say how the risks to you compare with David Spiegelhalter's normal life, say, a year or two ago?
David
Yeah, I did this and I was amazed, and it's helped me enormously in dealing with my own feelings about this. I was using the mortality rates, the infection mortality rates, published by Imperial College, which may be a bit high, but I used those. And they rise incredibly steeply with age - exponentially with age. And then I thought, you know, what else rises exponentially with age? Well, our normal risk of dying rises exponentially with age! And it's amazing - if you draw a graph of the average risk dying at each age in the country (you can get that from government life tables), and you plot these Covid 19 risks on top of it, they go almost along the line. It's staggering. And what this means is that, essentially if you get Covid 19, the risk of dying is very roughly equivalent to the risk you'd normally have over the whole year: it packs a year's worth of risk into the few weeks that you've got the disease. And that kind of puts it in perspective, what it shows is that this is a relative risk, whatever risk you've got, at the moment, it ups it enormously for that short period, if it's three or four weeks, then maybe that's 15 times your normal risk. And so we can really understand why frail, vulnerable people are at such high risk, why it's so dangerous for them because they're at risk anyway and suddenly it shoots up, they get a whole year's worth, you know, in less than a month.
Michael
So you say you say you find this reassuring!?
David
Oh yeah, because I like to think I'm very fit and healthy and I'm invincible! And I'm not that worried about dying in the next year. So I wouldn't get that worried about dying from Covid 19.
Michael
So if you're, say, a 60 something looking at the risk that you would have had of not making it from one birthday to another - that's about the risk squashed into a couple of weeks that you experience if you get Covid 19?
David
Exactly, very roughly. For adults over about 25,
Michael
So if it didn't worry you before....?
David
...Yeah..you shouldn't be that worried about getting the disease. But if you're actually old, vulnerable, frail, and realize that your life is limited, then this is obviously deeply worrying. And that's why of course, people, you know, are being shielded because these are the very people who, who it's going to affect.
Michael
Now, we should be clear, when we say it's the same as normal risk, we don't mean that it's instead of normal risk. You can't swap one with the other!
David
No, no, no, if you survive this, you still got to go through the whole year as well! So yeah, I'm afraid you don't become invincible for the year if you survive Covid!
Michael
Then we've got to adjust that, possibly if we say that maybe you wouldn't have made it through the year anyway. There will be some overlap with those people who are going to die anyway?
David
Yeah, there will be some overlap for very vulnerable people.
Michael
Can we talk about it in terms of life expectancy? Can we phrase it in that way?
David
I thought about that. Not really, I don't think you can do that easily. Because it's a short term impact. It's a real pulse of risk you're having to take. So I don't think it works well in terms of life expectancy. And it doesn't work well in terms of, you know, just a percentage chance of dying because it varies so much between people. It's almost pointless mentioning it. It just multiplies your standard risk by a certain amount.
Michael
For quite a lot of people I can imagine that putting the risk like that, it will suddenly feel rather unremarkable, even for some of those, you know, in a reasonable state of risk, you know, 60 something and they think, well, I wasn't afraid before I should maybe I'm maybe I'm not afraid now. Can you explain how that kind of risk can result in so much pressure on the NHS, if it seems unremarkable?
David
Oh, just because it's vast numbers of people. If everyone got it in the country, you'd have a year's worth of deaths in a month. You'd have a vast pulse of pressure, because all this risk has been packed in together. Spread over the year, we can deal with this risk.
Michael
Okay, finally, just looking to the endgame.... is your sense that unless a vaccine is discovered, we might be expecting the same number of deaths eventually? And what we are trying to do is just flatten that curve and spread them out so that we can cope with them? Or do you think we might actually be able to suppress it in some more permanent way?
David
An integral part of the modeling by Imperial College was the assumption that we could space out the impact of this virus by starting off with very stringent measures, then relaxing them and then expecting that there would be a resurgence and then having to bring in some measures again, maybe not the full lockdown, and so on. And so it forms a wave of bringing in measures and then relaxing them, in order to stop the hospital system being overwhelmed.
Michael
David, thank you for helping us get a handle on what all these numbers mean and for your own frank assessment of just how much we still can't be sure about. Glad to feel that you're not too worried. I have to say I'm not too worried either and I'm supposedly at high risk. For those who are worried, I hope there's at least some evidence in this podcast. Thanks very much indeed.
***
Views to share? Get in touch on Twitter @RiskyTalkPod or email riskytalk@maths.cam.ac.uk
Risky Talk is produced by Ilan Goodman for the Winton Centre for Risk and Evidence Communication at the University of Cambridge.
