Should women get routine mammograms? Should men get regular PSA exams? Robert Aronowitz of the University of Pennsylvania and the author of Risky Medicine talks with EconTalk host Russ Roberts about the increasing focus on risk reduction rather than health itself as a goal. Aronowitz discusses the social and political forces that push us toward more preventive testing even when those tests have not been shown to be effective. Aronowitz's perspective is a provocative look at the opportunity cost of risk-reduction.
United States


00:00:05welcome the econ talk part of a library of economics and liberty I'm your host Russ Roberts at Stanford Steve Hoover institute our website is econ talk dot org for you can subscribe to comment on this broadcast and find links and other information related to today's conversation but also
00:00:22find our archives we listen to every episode we've ever done going back to two thousand and six our email addresses male econ talk dot org we'd love to hear from before introducing today's guest I want to alert listers that there's a special postscript in the end of this
00:00:39week's episode where I reflect on some of the empirical issues that came up during the conversation so after I thank my guests please stay tuned for some thoughts on risk health and data that are really important today is October twenty second two thousand and fifteen and my guest
00:00:55is Robert around awaits his a position the chair of the department of history and sociology of science at the university of Pennsylvania in his latest book risky medicine our quest to cure fear and uncertainty is the subject of today's episode Robert welcome to econ talk great to be
00:01:11here what is risky medicine well I mean the term to cover a few different things that I I think the character is a lot of modern medicine %HESITATION wanted to risk centered medicine this medicine that's focused on reducing the probability from bad outcome as opposed to medicine that
00:01:33is any kind of medical intervention that's near to treat symptoms or change of passage you are school process for doing harm in the body %HESITATION and their number aspects of risky medicine that go along with that risk centeredness in particular he way in which we think about the
00:01:54efficacy you know how we know something works in medicine has shifted in many cases from you know seeing the disease disappear or symptoms get resolved to real people living longer lives to I'm looking for intermediate and points of reduced a group you know probabilities probabilities of some bad
00:02:15outcome happening you know you're you're better because your cholesterol is one fifty when it used to be two hundred or your blood pressure is one ten when it used to be one forty and and the third element of risk in medicine you know and this you know I'm
00:02:27obviously the bluebird playful with the title nuisance at some of the things are also dangerous but it's not my primary intention of holding the book risky medicine third element is the fact that we live in a world where there's much more profit to be made when %HESITATION pharmaceutical
00:02:45companies and caught a device makers and medical specialists develop interventions that reduce risk and what I mean by that is in the old days I write this manager that I have in the book from actually somebody else who talks about a nineteen fifties pharmaceutical convention of of different
00:03:11Kerr drug companies and someone got up and give a speech and said we've done really well with our new antibiotics but we have a very very bad business model we have products that immediately consume their demand to give people a better and they stop buying a project with
00:03:25a product he's got to figure out a better way forward to the present a better way has been figured out if you have drugs rejections that promise to treat risk people are possibly possibly the whole population could have even some small probability of a of a bad outcome
00:03:45and be the market for disease and they might need to take this drug or intervention their entire life so %HESITATION third element of what I mean statement is certainly the case with it in a world where often we're paying we're we're consuming products using other people's money third
00:04:07party payments in the medical area that combined with the profit motive leads to a pretty unhealthy a dynamic of %HESITATION pushing products that people think well couldn't hurt better safe than sorry in in your book to a large extent as you could call the better safe than sorry
00:04:26the dangers are better safe than sorry not a title but but that's really I think part of of what your what you're exploring here the natural impulse to human beings have to avoid danger and then the opportunity to have somebody else pay for a chance to reduce the
00:04:41risk that this early the actual effect of it which is what I think is is your deep insight were often not right getting healthier were quote reducing the risk which they're not the same thing are they they're not the same thing I mean it's been something of a
00:04:57complicated argument in that you know whether or not intervention works according to what we might think it was the highest interest scientific FC you know proven in a randomized clinical trials to improve lifespan or reduce morbidity %HESITATION I did that kind of evidence exists behind the practice is
00:05:17not mysteries reason why people take or doctors prescribe or for use in that direction is often an element of what I call psychological or social efficacy at work you know better safe than sorry capture some of that that is often things are done because you know they may
00:05:37have some you know stated objective benefit on health but under the underlying logic of wire the product is usually have another reason now is this is most clearly hate however obviously at the end in practices that there's lots of evidence that there's not much benefits so like %HESITATION
00:05:57routine fetal heart monitoring in the end end quote unquote normal labor unison lot of studies that have shown that there aren't really any %HESITATION a significant health benefits to you know we don't talk about this when you see us in the women's labor and the other a microphone
00:06:15goes on the belly and the heart rate is is %HESITATION is measured you have a continuous sort of feedback loop and it's very reassuring to people of course referring only to list started to sell reading it really gets scared but what about bracket that point for a second
00:06:32%HESITATION there's a lot of evidence that they really it doesn't improve outcomes at least a lot more see sections well yes there may be many many expert panels get up and pronounce you know this evidence and and recommend we must do something about reducing the amount of two
00:06:46sections and we we we should probably find a way of getting rid of routine fetal %HESITATION hardliners but it never happens okay and you feel like yeah it's I mean I've found that to be a very faceting example it's typical the number the examples of book there other
00:07:01different kinds as well but this particular kind on the surface there's no side effects from it is just it's harmless I mean you're not hurting the child or not during the the the feeders and similarly when you do a blood test over doctor kind there's no we already
00:07:15taking the blood so if you check for your health your prostate there's no there's no harm done there but the real side effect in this is I think one of the most powerful lessons of your of your work is it that leads to often a chain of events
00:07:29that's particularly unattractive although seemingly inevitable and it's funny you mention that the fatal heart monitoring our first child the heart rate dropped precipitously during my wife's labor and our main doctor had not arrived yet and the the internet there was there were %HESITATION said we have to do
00:07:49is he's after for C. section and we weren't very staff about that and but it is not a place of calm decision making in as well I don't know all there is to know panic in the air on everybody's part %HESITATION and including the the the doctor who
00:08:05was on on call on duty then and dove fortunately Dr rubbed in time before anything was done and said out while she she had a contraction in the heart rate dropped in us back to normal is fine and nothing happened we are very close to having a C.
00:08:20section that was probably not probably certainly not necessary and would have added additional to my wife yeah and out all that through a little bit if you have a good outcome which he section I hope you had a good outcome was a regular fractional to look %HESITATION it'll
00:08:40self reinforcing to court because you know things worked out well it it probably was the right decision you know we we don't tolerate a lot of cognitive dissonance and there's something I call the book three the elephant in green Shamus problem sorry well it's just a stupid joke
00:09:00you know I grew up in Brooklyn we tell these %HESITATION if I can help each other's sense of irony on the stage jokes that didn't have a powerful punch points but the story went you know what he says to the other wide well since we're green pajamas and
00:09:15the other kid says I don't know and often the person telling the story says camouflage swells and can camouflage himself and pool tables and the public has puzzled look and says what and the storyteller goes on have you ever seen an elephant on a pool table you see
00:09:32it works and you know there's you know in many cases the absence of something bad happening is punitive evidence of something works and %HESITATION somebody who's had a screening test and early it on the basis of that some early pre cancer diagnosis and then surgery and chemotherapy for
00:09:52that problem and lives to tell the story ten twenty years later feels the Dodge the bullet and %HESITATION you know they would see to it which could be true but that's you know that the the you know I'm a big proponent of it having really good evidence interested
00:10:11misunderstand my cynicism about a week actual decisions get made to be an argument for you know letting a thousand flowers bloom when I think it's a group to agree possible we we should try to find %HESITATION you know you can't you can't model every medical decision and there's
00:10:28a lot of these statistics in people's bodies and circumstances make the application of aggregate data anyway individual difficult but those things before you know aside it's really important to %HESITATION get evidence that you know I I guess you just to start in the beginning of your question a
00:10:43few minutes ago when you said you know it a blood test is only information or the heart you know the heart monitors and harming the baby kind of thing you know my I guess I've been senseless my here stands on end when I hear that kind of thing
00:10:56because it's like a just as you said it's not the information that the dangers of course in some things like mammography you could have some radiation risk but for the most part it's what information to us and how would how would trigger is like your intern in the
00:11:11mold and I almost had his way trigger some unnecessary intervention %HESITATION and I guess the other thing I've been the only thing about just information that is worrisome is that a lot of our unit is much the making an argument a lot of for screening test fetal heart
00:11:30monitor we just talked about serve the psychological fortune to control our fears reduce uncertainty you need to ask the question we're the fears and uncertainty come from in the first place you know not you know wheelchair destined to some parts of the human condition that cure disease but
00:11:47many things we do exaggerate or complicate those fears and support from those very things that also have a role in controlling fear so for example screening mammography historically is behind that it's it's the cost of in many ways the rapidly rising incidence of breast cancer diagnoses through the
00:12:07seventies eighties and nineties %HESITATION it detected a lot more cancers and many more people were treated and the prevalence of cancer increased you know at the end at that same moment people your work is just more of it too much everybody everybody everywhere you look cancerous near one
00:12:27of the antidote that is to go get yourself screens which produces more people you know what was the diagnosis so there's a kind of %HESITATION this is for a lot of screen tests and actually is through the development of a lot of public health programs around the use
00:12:42fear as a motivator in even the early part of the century this kind of self fulfilling prophecy or order catalytic reaction that she taught herself aspect of things that we identified in the book and post my previous work on breast cancer that for trouble says so the challenges
00:13:02and I and I face the same issues as you do in economics because I'm a big skeptic about some of the empirical claims that economists make in the precision of it and the science claims about it in and then people so you don't understand evidence we as I
00:13:16am I'm interested in you're not you're you're anti science and I say I'm pro science in favor of good evidence just not very bright bad evidence so the alternative when you say things like oh this this encouragement of mammography and and screening in its cross many other diseases
00:13:35that's led to this epidemic in some ways of cancer but there was there anyway and so a lot of people say well the alternative is just a okay so I'm not gonna get tested I'm I don't know that I have it it's better to better to be ignorant
00:13:49ignorance is bliss and of course that's not what you're encouraging either and it's certainly not a not a scientific attitude but I think that's for human beings supposed to quote scientists I think the real challenge we face is decision makers is is is between those two poles all
00:14:06I'd rather not know because it's going to lead to a bunch of awful stuff I don't want to have done that might not work first as well I might be at risk of death it's better to find out and solve the problem so you know the couple things
00:14:20first I think we need the best possible evidence there is which means investing in knowledge production the fourth clinical trials especially of new preventative measures and some type of discipline as a society to through insurance companies or government regulation or the morality of individual investigators to not just
00:14:43do something that just seems like just information is resorting about her like self evidently affective without much form %HESITATION you know which is in the way many %HESITATION preventive practices restitution practices actually been introduced in a kind of evidence freeway so I I think you know it's clear
00:15:00argument for a high bar of scientific evidence at the same time I wouldn't dismiss it dismissed the your head in the sand psychology of wanting peace of mind as some you know the whole of human psychology that shouldn't be listened to I think well when we talk about
00:15:18screening tests and active things that misses the sites were gonna find something people's bodies I think the actual Essex for the doctor patient relationship to the relationship of medical authority to to the late to lay people is is it is different than when you know somebody's abstractly ill
00:15:35with cancer and is desperate for a cure we're basically pushing we're recommending people come in for tests there otherwise healthy they're in a good state of mind potentially around it and there should be a pretty high bar ethical grounds as well I think for a week before we
00:15:51disturb people's peace of mind that we know what we're doing and we're not just creating %HESITATION unlike what we you know what a medicine that might respond in my body a little differ from your body you know sink for a cold or something I could have Masaki I
00:16:09could have some individual sense that even though one in general as this medicine there's no good evidence for it I think it works for my body there's a certain plausibility for that you know if however on scientific you might think that comes to risk reduction there is no
00:16:22like you know we're talking about nothing have symptoms to connections probabilities and it's just this case with a clear we want to really hard but hardcore scientific evidence and you know I guess you just as a personal anecdote %HESITATION my wife and I are both positions we had
00:16:40our son in born in nineteen ninety two and are we had decided that %HESITATION screening for down's syndrome was something we didn't want to do because we made decisions unlikely that happened we would go through with the pregnancy and we made a calculation based on looking at the
00:16:58data might my wife had been overjoyed for few years that the dangers of you know they were minimal benefits just routine offers of not risky in pregnancy they could find what we doctors call incidental illness you know things that have no importer can't do anything about as much
00:17:15as they would find anything that could be done during pregnancy plus also we ourselves which are important to the overall was right about that yeah yeah yeah and they will be made this decision which is going to go without routine ultrasound and I can't tell you how many
00:17:27times turquoise pregnancy we had a like I do also want you know that was coming to which we need to check the dates you know you know they they understood almost kind of a service like you know teeth whitening at the end of a dental visit or something
00:17:41that made people go home with a picture and feel good about themselves or you know information about gender or something and I'm not defending this this communities a credit decision my wife and I made but I think there's something real cute deciding that you don't want to medical
00:17:55eyes on some part of your life if there is a good data pro pro that medicalization you know I think it's important to find twelve people switch to do it which I don't think it could have been done from the outside we could avoid this offer since my
00:18:10wife and I hadn't been physicians and you know had the authority to sort of push people away well we did we did in some funny there for our first child was born in nineteen ninety two also we also made that same decision are we did push them why
00:18:24wasn't easy and hard it's because people look at us like we were crazy is like again like all your body only intonation consigna evil anti technology I eat your primitive don't you want to know what my answer was no I don't want to know when they couldn't it
00:18:40was puzzling to them partly because it was just rare %HESITATION but you don't and you know you and I talked before this I mean there are certain screening programs that I think of this pretty good data for %HESITATION in some situations colonoscopy if I'd had to so the
00:18:58disturbance to peace of mind thing I want to elaborate a little bit because I I don't think it's just a matter of in a situation you your wife and my wife were in with %HESITATION you know what normal part of life having a child and and wanting it
00:19:13to be as free of medical problems as possible it's also very very powerful this issue is also very very powerful driver of our overdiagnosis you know for treatment of some diseases and let me illustrate this with the problem of prostate cancer and I'll do the for the kind
00:19:28of political situation with a man goes into his internist or family doctor's office and often without any actual discussion of costs and benefits a PSA test data to quote unquote routine blood work and the out patient it's called the nurse to the actual PSA level was high he
00:19:48merely six might have cancer comes back for a visit gets referred to urologist urologist the pros and cons of the whole thing may ask for another test broadly and some are going to biopsy biopsy today involves often ultrasound indicted survey of the whole prospect we work sometimes up
00:20:07to twenty or twenty five biopsy needle specimens are taken and it's not a typical specially as men get older for one of those or more of those twenty five biopsies temple low grade cancer cancer is created by psychologist Michael Gleason score and their cities will great numbers that
00:20:28come up I mean I know this happens a lot more from coal by friends asking what to do and the the urologist have come around to the fact that there's not very good data %HESITATION supporting radical prostatectomy are pushing people to definitely of radical prostatectomy a radiation under
00:20:47the situation that is many people seem to let along just fine even notice from the call cancer in their bodies and picked up by screening test the alternative to going for surgery and I would be a little bit long winded here but I want to get to this
00:21:01point is not just walking away from urologist but you're now committed to sort of in most cases a lifetime of every six months getting your PSA tested again Wilson there's you know there's a lot of %HESITATION innovation in the surveillance routine for people get repeated biopsies repeated ultrasounds
00:21:18or the look at the %HESITATION free PSA or the rap or there's a complicated Norman Granz tracking the changing brakes of PSA levels and often this triggers some threshold which leads to surgery but the thing that I have noticed this kind of state of rest in a state
00:21:37of anxiety feeling like this sort of Damocles over your head and many people many men decide to get a prostatectomy not to rid themselves of live a longer life but the results of the state of uncertainty that they found themselves you know I guess it's not a trivial
00:21:55small thing not that your decision about not having noticed some small one but it becomes very consequential when the medical routines themselves create a kind of experience state of bodily state of bodily rest that involves routine unpleasant routines you know %HESITATION thankful visits with doctors contest and a
00:22:15very reasonable response to a lot of people is likely to be done with the get the thing out and not just that but the the family members are even more eager off and then the person with stadium with the problem because they're they're afraid to and there discern
00:22:31Demichelis hang over their head as well and and the way you describe is exactly the way I've heard it described by many many people in my in my family and friends when they have these issues come out just get out of there and unfortunately getting out of there
00:22:46aside if it has concept consequences it's it's not easy surgery yet %HESITATION book but more than that a lot of times it's just there's no point to it the you've got a slow growing cancer is but has that see in it skin is scary breast cancer you actually
00:23:02mentioned in your book I found it fascinating you said that you know like the ultrasound you've gone to your doctor to make sure when you get your physical make sure you don't get the PSA and the last my last physicals a few months ago my most recent physical
00:23:17and my PSA came back low fortunately but I asked my doctor who's a smart I respect him a lot he's a very good doctor said why'd you do the PSA he said I don't know you know it's because it's kind of like it's just that there's a box
00:23:30that is what gets checked it it's been jacked up and I'm gonna be more aggressive next time not to check it to make sure it doesn't get test I don't want to know that number because not a meaningful number %HESITATION yes I'm not putting my head in the
00:23:42sand I'm not I'm not being on scientific fact there's a lot of evidence that it's a good thing not to know that number it seems our listeners make their own choices consult with their own positions we don't give medical advice here but I personally will not be getting
00:23:57a PSA test anytime soon as a routine matter yeah and I think you know about a couple of caviar to you and I should pay attention to what's new scientific developments %HESITATION you know and understand that the situation is very different when a PSA test is used as
00:24:15part of a diagnostic routine because your doctor for example felt on a natural on a rectal exam you know there's a lot of subtleties to you know any any task in a way you know in in in the good doctor you don't have to go to medical school
00:24:33education but you you know you you should have a doctor that %HESITATION explaining herbal teas Q. you know you're not you're an economist and one of the things that I find interesting in understanding the sort of psychological dimensions of our here is about the you know behavioral economics
00:24:50literature and how it might apply to look at historically and one of the really interesting things is that this is not from risky medicine for my breast cancer book I I started the book with he suppressed a woman who had developed breast cancer we think it's what's it
00:25:05called probably was breast cancer and in eighteen twelve Philadelphia and it turned to her brother in law was the leading surgeon in America the time he was sick and didn't you know that you know them and there's a lot of services the cancer community twelve and it was
00:25:24a brutal operation done without anesthesia but not so much the dangerous people also didn't believe that you could actually cure cancer by surgery alone was really done but this is the sister in law it was seemingly small %HESITATION and they waited two days of confrontation about it and
00:25:41she left a whole slew of letters allowed me to serve in going into decision making but the thing that kept her which she explained to her father was living in England months later why she did this incredibly painful our you know implication for pressure that is featured on
00:25:58our kitchen table in Burlington Jersey he said in the end she would rather go to her death to talk include a few quick read since I'm not gonna get this right now with the book in front of me but she said she'd rather good we're getting along at
00:26:11all but we do with no stone unturned %HESITATION ceci had done everything she has no regrets before she died and you know the economy in the world you know powerful you're restricted fishing make me talk about the suggested regret you know this was this was and is active
00:26:33you know I'm reminded of the since the family members were also arrested the worst worth watching the most important interest group in these conditions to go for surgery you know I think you know at least you know I grew up in whatever the guilt is kind of a
00:26:48familial issue not just the individual issue and it and it's stating this kind of guilt or regret I'm not doing everything possible now and bad right and it is very powerful force again with their little before people knew what behavioral economics was I've been thinking about a lot
00:27:06lately about this the science of regret we we really be alike and to prevent it would like to prevent regret and yet there is no way to prevent because there's type one and type two errors there's false positives and false negatives and there's times we make we we
00:27:25act we act and think bad things happen we acting good things happen we don't act and good things happen we don't activate things happen and it's very hard we don't feel the same about active and passive actions very hard to to accept that and other errors of omission
00:27:41in the action yesterday very it's very tricky and very hard to be normative here and you know one of the startling things not for my own historical work but in college of my of mine who study the overdiagnosis no problem using health service research kind of actual techniques
00:27:57and surveys whatever is that one of the surprising findings is that people who've got approval for positive diagnosis of cancer and lived for weeks or months with the feeling that they had cancer they were later found out by some confirmatory tests to me before the time of operation
00:28:15that there was no cancer in the body these people do not generally end up being like you know the I got critically wounded for the advocate for doing less many people who find social situation are actually more pro screening and more pro intervention than people who weren't in
00:28:34my mind harm this way and you have to imagine I mean this is empirical survey data you have to imagine that there's some psychological condition where people feel like they had a like it had some spirit having dodged a bullet you know they got some exposure to death
00:28:49and they didn't do you know an extraterrestrial refusal was Justin said from since the fourth string that within removed and it has a kind of positive meaning to people and you know these are very much trivial thing he thinks about the core of a lot of work conundrums
00:29:06about had actually decide what are you know where to where to look for rest and what they do when or where or not and what to do about it yeah I don't mean to trivialize the but it it's a little like %HESITATION a roller coaster ride right you
00:29:19go on the road evicts give give great fun because at the end use arrived it and it's yeah thrilled that's that horror and thrill of fear but it's over soap we glad you went on the roller coaster oh yeah it was great so there is a similar emotional
00:29:36roller coaster there for that that that false positive are false negative in this case I guess %HESITATION not false positive but at let's let's go right by the where I've never gone on a roll yeah yeah can either be able to play in my sensitivity to this issue
00:29:55with someone so yeah exact opposite of what I grew up with what you know about a mile away from the road closed as corny I want to like you like horror movies no I don't either saying we got them coming to so maybe bigger each year you know
00:30:09the kind of whatever test which is due to people to help cut interest make risk decisions so I'm gonna I'm gonna go back up a little bit and I want to talk about this whole general concept of of risking as and it comes through in the book in
00:30:25a number of places obviously runs with it a whole book in many ways but I want to get at it through the chapter we talk about the Framingham heart study and I I argue and %HESITATION I take some Flack for for my from my listeners and readers that
00:30:40epidemiology is %HESITATION eye sight I've described it as an intellectual cesspool of which probably is not the most flattering way to describe it %HESITATION but there is a terrible problem in epidemiology as well as economics that return about complex systems we can't control for everything we're trying to
00:30:59isolate the impact of one variable or two variables and I'd like your thoughts on that input frame it in the end pardon the pun in the Framingham heart study is we're sort of this right this this phenomenon was born of of egg of of risk analysis ever the
00:31:15general population yeah so you know the next chapter is not meant to do you know damn the frame him to cater to no effect I had the great pleasure of you know almost all the leading investigators to study stored nineteen forty nine or long dead but I I
00:31:32did some research on this on the study while some of the principles rescue this world alive for all you know that do they want a card carrying epidemiologist they were clinicians in practice who died one reason other ended up in public health service marine hospital systems and we
00:31:50had this practical problem of the newly discovered that you know heart disease epidemic you know that because Eisenhower to have a heart attack while in office and seemingly was the white man's burden look at that stressed out middle class executives were falling left and right around them and
00:32:05and it was very little knowledge you know about the cost of it and they ended up %HESITATION not in a very pre planned way Kathleen this way and that and ending up with a very you know %HESITATION interesting longitudinal study of people who didn't initially a heart disease
00:32:23and following them for a very long time check to study continues in their children and grandchildren today to see who develop heart disease but the book but these were clinicians and they what they were looking for their audience essentially were the physician in private practice and what kinds
00:32:38of factors they could find the course of a normal physical exam and laboratory analysis that could help them predict you know who might drop dead of a heart attack or not and on those terms I think the study that we you know was remarkably successful and and I'm
00:32:52very consequential the way we think about heart disease but %HESITATION they understood the investigator said it's clear if you think about this for awhile that Kelly was all based on these individual factors on like what do you know how many cigarettes individual smokes %HESITATION and what their blood
00:33:10pressure was but there could be no way in in this kind of study of individuals in a way that was contrary to another community to understand the things that happened above the individual or super individually in you know in some sense one of the contributors to the mid
00:33:27century hard epidemic was the sale of tobacco and how to battle get at the cigarette getting everybody's body you know there's a complex story of marketing of but it made into that coated role for southern states Democrats in political economy you know you know complex story of factors
00:33:48you know but but the framing inflicted itself only study visual factors and and also with the resolution is the knowledge they had at the time so you know what me to frame him story is a story something against and %HESITATION the risk factor getting attacked the first time
00:34:05in the non actual real setting the term respect refused is in nineteen sixty two Framingham study you for birth a certain kind of mindset it has probably contributed her health benefits for sure up it's very narrow and very individual %HESITATION and in some sense the roads not traveled
00:34:23has been ignored you know they were in order for the Framingham study but it's a we've been so %HESITATION referred from the dark in the rodeo Turkey further away from it yeah and %HESITATION you know any other aspect of it is that these receptors became prior to the
00:34:39frame him study your doctors it okay but only advice to their patients or something but they they didn't see themselves as like you know diagnosing specific preventable factors and giving people medicines for them that was kind of hocus pocus and something you know it was the job of
00:34:55public health authorities or something not theirs but we saw this rapid change in a war so much of your visit to primary care doctors really about risk interactions you know and kind of epidemiological knowledge base became %HESITATION it a platform on which everything is built of course today
00:35:14universal idiotic I would call it you know intellectual for anything but many epidemiologists themselves especially around small relative risks we're in a burgeoning industry of people find that you know that some tiny factor in lifestyle or diet or your increases you know your chances and from statistically significant
00:35:35value but but whose impact you have no idea what it means you know and then the next day somebody else produces another study another observational study to go the other way you know into this kind of crisis within a chronic disease epidemiology very you know so much so
00:35:51that you know some people call for it unless it's something as a risk factor seven a relative risk of seven or something even bother publishing it because you know what is waiting for you know somebody else to market regarding yeah but you get on the you get on
00:36:05the front page of The New York Times for her to often to resist the temptation if you had a positive find it yeah not often if if the negative final act so yeah I'm bringing it in interesting historical moment %HESITATION and and you know that are you know
00:36:23I I did want to include in the in the in the book dot rather easily to a very neutral in the book I was waiting for a little bit more of them but it was more of a historical descriptive episode but I think it's it's the %HESITATION its
00:36:37effect on the site Geist is what comes through died this sort of yes general idea that we should be reducing our risk through exercise diet a lifestyle center not you know that that has become you know a pervasive aspect of our lives and our I let me ask
00:36:56you a few questions about that should I get a report back so I don't forget this night undergraduate your tent and I often ask people like your kids you know whether they're help your what health it and the response I often get is I'm healthy look I eat
00:37:12you know don't eat carbs and the old boy gluten out I go to the gym and whatever it is it's not like the term you know that the point of trying to convince me to settle for my own good but if these are not as I think of
00:37:26the means to help what people think it helps is and and we do have you know a Martian coming to you know the US in two thousand fifteen might find this odd yeah I know it's a great point so what what should my attitude be %HESITATION if I
00:37:45want to prosper and live long is a good to go to the jam is it's foolish to be worried about these things but you know I kind of get a physical trying to get a closer collaboration you know that you know the major things that don't smoke billowing
00:37:58avoid something extremely extremely obesity and and occupy trucks you know %HESITATION the %HESITATION those are the major things you you know you need to do this like I said a few a few screen tests in my fax nation for children the end of the record execs nations are
00:38:16or pretend to have I mean in terms of secondary prevention there are some things like beta blockers after heart attacks which is shown good evidence for but you know the majority of like livestock claims that people make %HESITATION you know are not terribly well substantiated and you know
00:38:31look at some historical arc now people are complaining that the %HESITATION Kobe feed them because the risk is a response to the earlier dietary consensus that fact should be avoided you know and you know what to be cynical here in our town but so far no so as
00:38:49we go forward we're at this I think apparent cause of incredible explosion or knowledge about our own bodies through yes I don't know how the stick the confluence of the smartphone to destroy evolution and the dad big data of the genetic mapping and the costs coming down it
00:39:10we are standing on the it seems on the edge of a huge increase in knowledge about our bodies work do you think we're going to make some progress in those areas where we can actually make some reliable claims about lifestyle diet cetera because I my view is we
00:39:26know remarkably little right now that we know remarkably little radio okay has a kind of cautionary tale aspect to the new personalized medicine in Connecticut no knowledge of king of genetic risk you know that like that has already been flowing but it's likely to become much more prevalent
00:39:44in the course of a whole body screening happens and you know the nightmare vision I have is that you know when you get back %HESITATION twenty three in me results that say you have a three times risk of diabetes and a two times risk of heart disease in
00:40:00a yeah and lower than average chance of dying of testicular cancer or something that this will create a market for all kinds of it in a special in the risks are higher than normal for promised interventions that are based either on the genetic manipulation or some you know
00:40:17temporary astrological understanding that's going to be uprooted in over a week from now it's just very hard to respond what's one believes ones at risk or something and one of the one of the really tough implications I think and I mean we can talk about this a little
00:40:32bit of the dangers of the knowledge without %HESITATION good evidence the interventions being affected is whether we should have some threshold for not communicating this knowledge to people to you know we can do something with it and that does sound very what I like and you know and
00:40:49I don't claim to have a crystal ball to know what insights about risk will be fruitful and what not so I would not really want to be the you know knowledge or turning on the status of who gets research funding and not but the clinical and you know
00:41:05that and what information gets communicated patience I think you know project potentially significant avalanche of actually actually get information for sure isn't evidence about those actions that we might need some kind of %HESITATION in a worldly and bargain with ourselves to keep our heads collectively in the hand
00:41:23until we have good knowledge about particular interventions efficacy and I I don't know how that would work practically zero pressing on it level but I do worry about %HESITATION you know the process to be made the interferon uncertainty that will be that will be unleashed by having this
00:41:41kind of information communicated people without any good sense of what to do about it mom I think you're alone you noticed that a lot of problems with the data about risk itself which will get you know you know you know what changes with %HESITATION new resolution technologies and
00:41:59you know on selected populations and things like that but even if the knowledge was solid about probabilities without any direction to do something about it I'm not sure we really are doing people up ourselves a favor by well I think your book Yemen maybe this conversation is gonna
00:42:18help people think about how to think about this and what they want to know what they don't want to know and I I I just was it's a particularly appropriate weight to be having a conversation and there's been a recent change this thing was this week or last
00:42:30week over the frequency of of mammograms at the recommended and if not I'm not a woman but I love my wife so I'm very aware of the risks and and %HESITATION the questions of whether member regular mammograms are good idea but having said that I never looked at
00:42:50the numbers and because this week happened with these changes are recommendations because I was reading your book getting ready for the interview I happened to look at some of the the data that people are putting about mammography in fact I tweeted a Mother Jones article doesn't happen that
00:43:08often tickets to an article from other jobs but there is an article by the wrist and then I found another of peace are freer where was from on it and I was RD was a gem article I was stunned I think the jam article I think it said
00:43:23out of ten thousand mammograms there were six over six thousand false positives ended ten deaths or averted from those I mean it's it's a it's a stunningly imprecise here tragically imprecise thing and and %HESITATION I it just die I was shocked when you look at again I'd buy
00:43:46point here is you have to look at the numbers and one of the themes your book is that the culture what's in the air this sort of what's expected whether it's the ultrasound for the pregnant during the pregnancy or the PSA test it's just everyone's first of course
00:44:01you're going to get a mammogram well you don't did nothing I don't have the numbers to my fingertips and the numbers needed to treat statistic that you quoted gets better for women as they go through menopause and older %HESITATION you know but you're right Republic from I don't
00:44:19know how many thousand women years for screening you know to lesser amounts thousands of years a woman screaming to save to avert one death from breast cancer clustered many false positives which will lead to over treatment with himself Kerry death risk yeah it's not the same was saving
00:44:38the lives that went to all the others costs that come with it which are just part of like pushed to the side yeah and you know many many people have made you know observations and one of things I wrote the book for was to say I don't think
00:44:55another study another piece of data is going to you know change the game very much %HESITATION and I think we have to think about what works does screening mammography do for the different interested actors and by exploring that work and and historical construction conditions that make that work
00:45:15possible maybe there's another way out of the situation and and then you know a part of me says really the issue is not screening mammography screening genetic tests of things that you haven't yet haven't put and acts have not yet been put in the water because I think
00:45:32you don't want something which is a kind of equilibrium in people with you know it's part of being an American woman today go for their annual mammograms part of life in a way you know once a normal expected to stab him because of the substantial psychological work of
00:45:45rejection fear patrolling a certainty heart dislodge and so can my real hope in some way to prevent in a way to prevent more things being done outside experimental trials %HESITATION but you know it's very tricky you know what I mean we should talk in those two men talk
00:46:03about prostate cancer could hurry now to situations like the %HESITATION nineteen two thousand and nine I'm some twenty years after my pastor PSA testing used widely in American medicine in society and the rest of the world as well with only two thousand nine they were the first results
00:46:21of randomized control screen yeah shockingly consisted Oct so I mean the real fact is not like what the study show it's that you know this thing can amass phenomena with its own inertia and social psychological FC long before there's any scientific evidence interest a bit player in the
00:46:38story but but let's forget about that for a second and look at the data itself and one study at the time showed actually no benefit for you immortality benefit for screening which in you know in some ways he would leave the state of Texas focus long enough and
00:46:54later followed complicated this result %HESITATION you know you should just not do it the other study was a multicenter European study where she did some subgroup analysis including some countries not others there did seem to be a better mortality benefit perverting prostate cancer deaths but it's a very
00:47:11high cost and the the rule of thumb number needed to treat statistic that was quoted by the editorialist and the authors themselves of the study was that some fifty men would need to be treated forget about screen you know thousand ebony to be screened at the fifty men
00:47:25who can end up being treated but we have too many to be treated for prostate cancer it was picked up by screening in order to avert a single desk and a large numbers of since changed they've moved but you know it's very hard psychologically for individual sitting in
00:47:39the decision making it on the decision making seat to sort of put the rafter hands around what that might mean you know on that I mentioned and medicine I see lots of people %HESITATION end up with incontinence and impotence after surgery or to help them a blood clot
00:47:54near lake water hospitalized incidently and die in a way you know my own common sense you know it doesn't add up that I would risk you know sixty to one odds of getting this actually go in and you have to know have you noticed the man would have
00:48:10to have the surgery in order to save the life of a government must disclose which might make sense for someone else you know and also I don't really think so if they really understood what these what the risks were involved in some ways and usually another thing which
00:48:26I think we get to these really kept really difficult eleven the mantra of many of my well minded clinicians colleagues NSS is to say well what the teacher decides the doctor's side some kind of idealized model shared decision making and I we have a great solution herself but
00:48:47I'm fairly cynical about that being the last resort will be say when we don't have good enough evidence to actually resolve a policy or clinical problems to say equipment which you know in the information out let the doctor patient aside I just think positively it's too complex and
00:49:06may be just what I was saying that knowledge shutting off at least marshalling the knowledge production because in some ways maybe there are situations where the data is so confusing and we really don't have a clear idea to use some kind of principle first do no harm and
00:49:20not bring it up in the first place at all and not make it a misspelled word of what take the test and then have a shared decision making around it and part of this I have to tell you is informed by my really negative experiences of practicing permit
00:49:35for physician the nineteen nineties when all the major media which actually for many major expert advisory panel from different physician groups suggested either that women should get hormone replacement therapy when there %HESITATION menopausal or at least usually she had a discussion of the risks and benefits of giving
00:49:58this therapy and my analysis of the many observational trials that seem to show some efficacy to this thing we're all flawed in the same direction of healthy women affect there had not been a randomized control trial it was so much prosecute made and crass manipulation of the market
00:50:18by the hormone we manufacture the very term hormone replacement therapy was basically you know one of these terms that you know creatures on demand but you know if you ever can offer hope to hell it you know like you know you could call it you know just the
00:50:32names of the drugs or something else anywhere suddenly I had this US reluctance and I did not initiate discussions with my they brought it up I talk to them about it because I didn't think it was a good reason to do it and that was part of what
00:50:46I considered my medical responsibility artist albums of other things and not being pushed by special technically brought up there strode out into the ether they're not being discussed why this and it's one of the few cases in in in my clinical life it you know observers medical developments
00:51:03within there is you know unbelievable what the right answer that came out informed clinical trial the clueless health initiative that showed that the hormone replacement therapy by really given to prevent osteoporosis and heart disease you were given for the menopausal symptoms when they're given is prevented this do
00:51:20more harm than good which incontrovertible here much with that Woody Allen movie where two people are arguing about the meaning of Marshall McLuhan animal products along to begin theater and Marshall McLuhan comes you're right you're wrong you know like it's just rare that this in some ways so
00:51:38I think woody Allen's as turns the camera that points as why can't life be more like this so get your yeah well that that women's health initiative results reflect a little little moment like that around and having lived through that in some ways you know and having some
00:51:52you know historical research under my belt to %HESITATION the way we deal with uncertainty in the present you know could often be very either laughable or %HESITATION look back to some you know sure fire children or this Martian I invoke here and there you know with some degree
00:52:09of you know you know alarm and and and and you know how to keep what do people do that when you mentioned the idea of the people talking over the doctor and come to a decision it to be honest what we're really doing there's almost when there's no
00:52:24evidence it's almost like saying flip a coin we would never would be very hard for most able to say to get the surgery now well flip a coin odd that's an even if it's not a fair coin it's an unattractive way to make a decision my only thought
00:52:38about that mutual decision or whatever you want to call it is it is a mess aspect of talk therapy at least at least having discussed it may be a person would feel better than actually just flipping a coin %HESITATION I'm interested in what kind of response you've gotten
00:52:53from your fellow %HESITATION physicians from these kinds of arguments that you make did they see you as dangerous and I just wanna say by the way before I just wanna get the sand is truly important those people listening in that it's a it's a very interesting book risky
00:53:08medicine I encourage you to to read it and more than anything else when you hear this conversation I what I want I'd like blisters to take away from it is educate yourself look at the numbers yourself if you can't look at them yourself get somebody who's if you're
00:53:25not skilled enough for or you don't know enough to look at them get somebody who's topical because I'm a friend of mine asked me if I she's facing %HESITATION hysterectomy and she wants him as she could she get her ovaries removed at the same time and she sent
00:53:37me a study she found that she wanted to educate herself in its %HESITATION a recent study that found that %HESITATION removing the ovaries was dangerous because it led to an increase in the risk of heart disease and dom I looked at the dead at the study and it
00:53:53didn't look like a very good study to me because it had this problem that you're talking about that I don't know enough about the the the nature of the other women who made those decisions on where they're like the women who did not cite a clinical clinical trial
00:54:06and so I said that's not a good study that one and and she's a but it's a bit their loudest it had a big population I said that's not enough so yeah educate yourself think about it talked as people who are who are who have thought about data
00:54:20and try to get and try to help make the decision with evidence rather than just what everyone tells you because it's so it's complicated your question was how to my Christian college running back to that okay it's okay you know I haven't lost any physician friends and maybe
00:54:40you know like attracts like and you know I I'd like to Pennsylvania and I you know look Elizabeth brought high powerful service researchers were just what I feel deeply to evidence %HESITATION and %HESITATION most people realize that I'm actually providing a backstory %HESITATION have to do with social
00:55:00economic and structural context for things that are puzzling and you know the troubling in the epidemiological trends to do the dirty water just look in terms of data itself so I don't and I would say World War I I have given my grand rounds to you know audience
00:55:19of people who largely comply with most most scripts from screen recommendations and I'm not in favor of or whatever I get occasional person who it anecdotal story about how someone you know my first retesting got it was found at him for increased cancer and they were taken out
00:55:38in their life and if they hadn't had the screen test would be dead so there what do you see now %HESITATION and you know very rare you know I mean specially younger physicians you know whatever or are you would expect there to clean the cages in the Ciskei
00:55:55way to think about scientific evidence I went to medical school at Yale I wasn't trained that way are really but I think their lived experience of it out on the street doctors talk about incidental omits and they have some intuitive knowledge of how one thing one damn thing
00:56:10leads to another and some reluctance to go a certain way because of that and I think on what some of you know Michael Coen oriented sociology friends would ever go on a kind of direct frontal attack on the false consciousness of medicine and people is already I try
00:56:27to tap into the lived experience of you know patient friends but also %HESITATION doctors and try to give a kind of %HESITATION scaffold hang this kind of uncertainty and discomfort let your doctor who said you wanna know what do the PSA difference kind of doing it that's not
00:56:44an easy thing to live with you know when you're on that medical into it you do kind of what you're hoping you're actually doing some good for people you know or not not complicating it so I think if you you don't more of a message to give to
00:56:59the you know people outside of medicine the one which medical audiences without turning them off in a way is to realize that you know these easy these contradictions are lived out every you know every day in and offices of people in practice and they are subject to all
00:57:12kinds of undue influence is there trying to sort of secure from past between you know different shoals they're better you know crashes the ships about to crash into and you you know so I I've I've you know my good days I feel like I some social historical context
00:57:30for this thing and I hope the book is doesn't appear shrill you know on there are things that you know so much because we just don't know and %HESITATION are and you know I'm not by any means against in fact I'm very pro you know that thick or
00:57:47even the bio technological advances in medicine today are you know as long as we're objecting but we find to to local trials and and good evidence %HESITATION I have a good friend being kept alive right now by some are targeted therapies that just developed in like a phase
00:58:08one trial and %HESITATION you know I'm thankful I'm living in collected over the population affected for the individuals affected this is really you know some wonderful things to come out of you know the US and western world's commitment to biomedical research it's just that you know you don't
00:58:26throw the baby out with the bath water you know like this is a lot of problematic since the pharmaceutical companies to make one of the reasons I got interested in that that one of those problems and started the interview with about the the the profit motive to treat
00:58:38rest president symptoms or disease is I literally had a CD over very very very big maybe one of the biggest pharmaceutical companies we just like left his job like the week before talked to my students and you know he told us very cynical story said the street you
00:58:55know he meant well street %HESITATION requires a ten percent return on your increase in in sales every year you know we haven't had a new effective drug in in ten years you know we're just gonna come from and he said you know ultimately you can expect from this
00:59:12committee would have the kind of startup culture to develop new things they had a like basically by the patents of other people's trust what they were really good at was detailing physicians yeah and getting people to use medicines and so we are while there are indeed in the
00:59:31RD was you know impossible to separate from marketing frankly was to develop drugs against restaurant and comments are completely accurate you know things you know completed everyone has put drugs that will be for everybody and for life yeah so in a given that %HESITATION that's what I'd like
00:59:51to see a world of medicine with less third party payments more out of pocket we seem to be moving in the other direction but that's some that's another topic let's close let's close with some with your thoughts on medical education as you said you were trained up an
01:00:10acceptably good medical school and yet you were exposed to a lot of these kind of ideas %HESITATION I find it remarkable how few doctors understand statistical issues are and taking a statistics class or a buyer statistics class is not sufficient what you learn in those classes typically is
01:00:32a the definitions of the different techniques and now the tester Ron and but we aren't we don't give our we don't get many people let alone doctors are much I'd much of a training and this and what we would call risk analysis the roads the kind of thoughtful
01:00:47tradeoffs that we're talking about and when he mentions in a circuit wrenches survivorship your friend being alive on it the whole issue that runs through your book is that there's a quality of life issue here the typically gets totally lost at how do you train should we train
01:01:02physicians do you think it's a good idea more %HESITATION effectively in these issues well I'll give you two answers that maybe you'll be filled skirting the question but one is this is an issue for me since so many of the things we're talking about our master directions that
01:01:18everybody has to sort of make a decision about away and this is really just one piece of input to the other they're not even the gatekeeper for many of these things %HESITATION it initially came to public as much as you know healthcare workers you know %HESITATION yes their
01:01:33physicians rather was so up to the second thing is that you know how I vote my feet which is what I think I do %HESITATION in my day job is I'm sure history sociology science when we have these very large majors won the largest majors at ten call
01:01:49help these and it's getting people not at least you know or or maybe thirty of the very large undergraduate group organ into being physicians but the others going to the healthcare industry in their own defense to public health or who knows what they're doing and in the rest
01:02:05of us the starting display at some level and I think you know it intellectually what mind opening part of people's lives like undergraduate life should be if you can have a series of gauging with not just distance but the humanistic and other social sciences that you know talk
01:02:23about prepared of health systems and I you know the development of the history of therapeutics or %HESITATION history public health associated with numbers quantitative things but also to her local development of it that we're kind of I mean you know you're not impeachable kind of vaccinating these future
01:02:42doctors and healthcare workers and people the health system can be the consumers of what would later gonna happen during medical socialization with a jaundiced eye or at least a skeptical eye and a way of some vocabulary which to make such an experience that happened to them definitely focus
01:02:56away from my own energies in any part of it has to do with how credibly demanding knocks me you know medical schools but you know the real treating you know some of the formation of people's like medical personas happens when they do the residency training the US you
01:03:10know in the in the years of internship and residency were you know it's often you know maybe it's for the better now than the bad old days are treated where this you know people are pushed to the very extreme of their physical capacities and overwhelmed by things and
01:03:23it's just not a it's not a time of great reflection %HESITATION overall so but my own effort into doing so many things rather than you know being another person saying the medical attention include my little things %HESITATION which you know I go to some critical crucial making selected
01:03:44pretty boring but you know what I always feel like you know to that cliche of rearranging the deck chairs on the Titanic in how it's just not it's just not %HESITATION too much work is being pushed on it are in some way so having said that I'm you
01:03:58know what I can for my college teach little political at the %HESITATION subsections and I'm sure some of things to get caught you know one of the you know the most horsing about the post of the medical school training is the fact you're dealing with real people and
01:04:12real conundrum you know so you have some way of helping people process because experiences %HESITATION you know people have been harmed by you know you know medical overdiagnosis go for treatment in a way and and getting some language to make sense of it would probably be of help
01:04:29but I don't have a great for my own at the moment I guess it is but Robert around awaits his book is risky medicine our thanks for being part of a contact out my pleasures for talking to you now for a brief postscript on the econ talk over
01:04:46station I had with Robert around our wits about his provocative book risky medicine in the middle of this week's conversation I made a reference to the evidence on the efficacy of mammograms and we're talking about testing generally I got a little fired up and I think I pleaded
01:05:01people check out the data check out the evidence would you consider getting any sort of diagnostic test some my reaction in my passion there was to the philosophical issues the robber Ronnie which raised in his book are human desire to reduce risk apart my reaction was due to
01:05:18some reading I done in advance of the interview the Mother Jones article I mentioned in a jam article that mentioned the ten deaths were averted for every ten thousand women getting an annual mammogram from age fifty to fifty nine ten deaths overdid it struck me as a small
01:05:35number compared to the other human cost of regular screening sixty one hundred false positives over nine hundred biopsies that showed nothing but because women have anxiety and their family in addition there are nontrivial numbers of overdiagnosis lead to unnecessary mastectomies then there's the risk from the radiation and
01:05:56so on so the evidence in the chart in the jam article that I was looking at seem pretty important consider in a culture where until the recent changes in recommendations an annual mammogram was street is a no brainer released regular mammograms to looking at the evidence seems like
01:06:11a very good idea and that's what generated my passionate statement about look at the evidence look at the data member exactly what I said was something like that and I had that chart in mind so after I finish the conversation the recording I shared these thoughts with a
01:06:28friend of mine whose an OBGYN and she was not nearly as impressed as I was with a jam a charred and summary of the costs and benefits of %HESITATION regular month mammograms for women between the ages of fifty and fifty nine he pointed out for example that the
01:06:43measure of deaths overdid probably was based on older studies when mammogram technology was less effective also wondered about how they measured over diagnosis survival rates within without mammograms and I started thinking yeah how did they measure them and she wondered if the results distinguish between the average woman
01:07:02and women who maybe have a higher risk of breast cancer such as women are breast cancer in their family so that remind me to go back and look at the chart I've been reading and see what the source for the numbers or were I had stupidly treated it
01:07:16as something of a census of excess and counting rather than a set of estimates so I went from a supporting article is another jam article is from the journal of the American medical association and discussing mortality the authors say they got those numbers from eight large randomized deserve
01:07:32the death verdict of ten they got those that measure from eight large randomized control trials between nineteen sixty and nineteen ninety and then they mention the following quote some argue that the RCTs surround much control trials some argue that the R. C. treat RCTs are unlikely to be
01:07:53apple apple couple to women undergoing training today because they proceeded treatment advances that have powerfully influence breast cancer mortality in his older mammography techniques however the RCTs never less provide the best data available close quote well in that drew me up short of my friend was right maybe
01:08:14those estimates of tend as subverted were understating what the effects are today and by the way I put links of course up to the chart and the in this article I mentions the source for the chart in the likes of this episode so that's the does averted issue
01:08:29what about the other costs are the human health costs of a regular screening of course the rest of those estimates had issues as well are now some of them might be reliable but it's hard to know what I should have gone back and looked at the sources that
01:08:44they used to generate those estimates that I hadn't done that so maybe I was overly negative having looked at that chart dimensional this for two reasons the evidence is almost never straight forward it's almost always complicated second point is that it's hard to stay bias free I like
01:09:01to think of myself as a skeptic and I am a skeptic but I can struggle to be skeptical about my skepticism I think was a little too eager to embrace Serrano it's a skepticism about regular testing now that was going on in my head my word some of
01:09:15that skepticism as of his skepticism may have come out the conversation I may have been too strong I might have encouraged some of you are worried to think that not that the evidence is more black and white than it really is going to make it clear here looking
01:09:30at the evidence is always a good idea but the evidence is almost always murkier than advocates on either side of initial concede overlooking a complex issues such as health or economics for that matter the numbers really speak for themselves they're always questions of interpretation leaps of faith in
01:09:47trying to measure some variables along with the issue of confounding effects from additional variables that often go on measured inevitably assessing risk is very complicated thanks for listening this is econ talk part of the library economics and liberty for Mari contacted econ talk dot org or you can
01:10:10also comment on today's podcast can find links in readings related to today's conversation sound engineer free Cantacuzino yet I'm your host Russ Roberts thanks for listening talked to you on Monday

Transcribed by algorithms. Report Errata
Disclaimer: The podcast and artwork embedded on this page are from EconTalk: Russ Roberts, Library of Economics and Liberty, which is the property of its owner and not affiliated with or endorsed by Listen Notes, Inc.


Thank you for helping to keep the podcast database up to date.