WEBVTT
00:00:09.519 --> 00:00:12.720
Hello to all of my fellow travelers out there.
00:00:12.960 --> 00:00:14.560
I'm glad you're here.
00:00:14.880 --> 00:00:19.359
Wherever you're listening from, thank you for spending time with me today.
00:00:20.079 --> 00:00:22.000
Let's dive right in.
00:00:22.480 --> 00:00:26.480
Yesterday we saw that doubt can be manufactured.
00:00:26.719 --> 00:00:32.960
One industry wrote that doubt was its product, and we learned three questions to carry.
00:00:33.280 --> 00:00:34.880
Who is telling me this?
00:00:35.119 --> 00:00:36.479
Who paid for it?
00:00:36.719 --> 00:00:39.600
And what would they gain if I believed it?
00:00:39.920 --> 00:00:44.079
Today we asked the most basic question behind any number.
00:00:44.640 --> 00:00:46.399
Who was counted?
00:00:46.719 --> 00:00:55.920
In 1936, a popular magazine called the Literary Digest ran the biggest election poll America had ever seen.
00:00:56.159 --> 00:00:59.200
They mailed ballots to 10 million people.
00:00:59.759 --> 00:01:02.320
2.4 million sent them back.
00:01:02.880 --> 00:01:05.439
2.4 million answers.
00:01:05.680 --> 00:01:07.439
The result was clear.
00:01:07.760 --> 00:01:14.239
Elf Landon would beat Franklin Roosevelt with about 57% of the vote.
00:01:14.719 --> 00:01:21.920
Roosevelt won with about 62%, one of the biggest landslides in American history.
00:01:22.480 --> 00:01:27.120
How could 2.4 million answers be so wrong?
00:01:27.439 --> 00:01:38.000
The magazine had built its mailing list from phone directories, car registrations, and magazine subscriptions in the middle of the Great Depression.
00:01:38.319 --> 00:01:45.280
People who owned phones and cars and subscribed to magazines were not a fair picture of the country.
00:01:45.519 --> 00:01:49.599
And the people who chose to mail back a ballot weren't either.
00:01:49.840 --> 00:01:55.439
Meanwhile, a young pollster named George Gallup took a much smaller survey.
00:01:55.680 --> 00:02:03.840
Far fewer people, but he worked to choose a cross-section that looked like the country, and he called the election correctly.
00:02:04.159 --> 00:02:09.439
A huge number, the wrong people, a small number, the right people.
00:02:10.000 --> 00:02:13.120
That's today's lesson in two sentences.
00:02:13.360 --> 00:02:15.120
Why does this matter?
00:02:15.439 --> 00:02:20.000
Because every health claim we hear is based on some group of people.
00:02:20.319 --> 00:02:28.800
Every dose, every guideline, every study show, if that group looks like us, the answer may fit us.
00:02:29.039 --> 00:02:31.520
If it doesn't, the answer may not.
00:02:31.840 --> 00:02:36.000
And most of the time, no one tells us who was in the room.
00:02:36.319 --> 00:02:45.199
Many of us have had the experience of a treatment that was supposed to work and didn't, or a dose that was supposed to be right and wasn't.
00:02:45.439 --> 00:02:47.520
Sometimes the reason is simple.
00:02:47.840 --> 00:02:50.400
People like us were never counted.
00:02:50.800 --> 00:02:52.319
Here's where I stand.
00:02:52.639 --> 00:02:57.520
I believe medicine tends to be designed around whoever was counted.
00:02:57.680 --> 00:03:02.159
And for a long time, the people counted were a narrow slice of humanity.
00:03:02.400 --> 00:03:08.639
When you weren't in the study, you're being asked to trust an answer that was never tested on you.
00:03:09.120 --> 00:03:11.599
That isn't a reason to reject research.
00:03:11.759 --> 00:03:16.560
It's a reason to ask every single time who was in it.
00:03:16.879 --> 00:03:18.639
And I'll say something else.
00:03:18.960 --> 00:03:20.719
Size impresses us.
00:03:20.960 --> 00:03:23.199
A study of 20,000 people?
00:03:23.680 --> 00:03:25.280
Millions of users?
00:03:25.599 --> 00:03:29.199
But size never fixes a sample that's tilted.
00:03:29.759 --> 00:03:32.319
The literary digest proved that.
00:03:32.639 --> 00:03:36.960
More of the wrong people just makes the wrong answer louder.
00:03:37.360 --> 00:03:46.400
In 1981, researchers invited male doctors across the United States to join a study on aspirin and the heart.
00:03:46.719 --> 00:03:49.520
More than 22,000 men took part.
00:03:49.759 --> 00:03:52.319
It was called the Physician's Health Study.
00:03:52.560 --> 00:03:55.520
In 1989, the results came out.
00:03:55.840 --> 00:04:01.039
Men who took aspirin had 44% fewer first heart attacks.
00:04:01.280 --> 00:04:07.520
The finding was so strong that the aspirin part of the study was stopped years early.
00:04:07.759 --> 00:04:13.840
It was a landmark, and for years it shaped advice about aspirin for heart health.
00:04:14.159 --> 00:04:16.240
But notice who was counted?
00:04:16.720 --> 00:04:18.959
Men, only men.
00:04:19.519 --> 00:04:26.000
It took until 2005 for a large trial to answer the same question in women.
00:04:26.399 --> 00:04:34.319
The women's health study followed nearly 40,000 women over age 45 for about 10 years.
00:04:34.560 --> 00:04:38.720
Half took low dose aspirin and half took a placebo.
00:04:39.920 --> 00:04:44.480
For women, aspirin did not lower the risk of a first heart attack.
00:04:44.720 --> 00:04:49.759
The numbers were almost identical, about 1% in each group.
00:04:50.000 --> 00:04:57.120
It did lower the risk of stroke a little, but the heart attack benefit seen in men simply wasn't there.
00:04:57.600 --> 00:04:59.120
16 years.
00:04:59.439 --> 00:05:04.639
For 16 years, women were living with an answer that had been found in men.
00:05:05.199 --> 00:05:08.079
And notice one more thing about the men's study.
00:05:08.319 --> 00:05:14.639
Before the main trial began, more than 33,000 doctors started a trial period.
00:05:14.879 --> 00:05:20.000
Only about 22,000 went on to the rest of the real study.
00:05:20.240 --> 00:05:25.600
The trial period was a way of keeping the people most likely to stick with the pills.
00:05:25.839 --> 00:05:29.360
That's a common practice, and there are reasons for it.
00:05:29.600 --> 00:05:36.560
But it means the study counted people who had already shown they could take the pills and who stuck with them.
00:05:38.240 --> 00:05:44.560
When a sleeping pill met a woman's body, here's another story about who was counted.
00:05:44.879 --> 00:05:56.399
For years, a popular sleeping pill called Ambient or Zulpadem was prescribed at the same starting dose for men and women, 10 milligrams.
00:05:56.639 --> 00:06:06.720
Then in January of 2013, the FDA told the makers to cut the recommended dose for women in half, from 10 milligrams to 5.
00:06:07.279 --> 00:06:07.920
Why?
00:06:08.639 --> 00:06:12.560
Because women clear the drug from their bodies more slowly.
00:06:12.800 --> 00:06:21.759
That meant some women were waking up with enough of it still in their blood to impair them the next morning, especially behind the wheel of a car.
00:06:22.000 --> 00:06:29.040
Women had been taking it for about 20 years before the dose was cut in half for them.
00:06:29.279 --> 00:06:32.240
Think about what that means in a real life.
00:06:32.480 --> 00:06:37.519
A woman takes the dose her doctor prescribed exactly as prescribed.
00:06:37.680 --> 00:06:42.399
She sleeps, she gets up, makes breakfast, and drives her kids to school.
00:06:42.639 --> 00:06:48.160
She has no idea that her body is still carrying enough of the drug to slow her reactions.
00:06:48.480 --> 00:06:50.079
She did everything right.
00:06:50.319 --> 00:06:53.839
The dose was simply built on the wrong body.
00:06:55.279 --> 00:07:00.000
And it wasn't an accident that women were missing from so much early research.
00:07:00.240 --> 00:07:09.199
In 1977, the FDA issued guidance that kept women who could become pregnant out of the earliest stages of drug testing.
00:07:09.680 --> 00:07:13.360
That guidance wasn't changed until 1993.
00:07:14.480 --> 00:07:19.519
Sometimes the problem isn't who was counted, it's how they were counted.
00:07:19.759 --> 00:07:26.399
For years, Colgate ran ads saying that 80% of dentists recommend Colgate.
00:07:26.720 --> 00:07:29.680
80% sounds like a landslide.
00:07:29.920 --> 00:07:36.639
In 2007, the advertising watchdog in the United Kingdom ruled the claim misleading.
00:07:36.959 --> 00:07:37.600
Why?
00:07:37.839 --> 00:07:44.319
The survey had let dentists recommend several toothpaste brands, not just one.
00:07:45.040 --> 00:07:51.680
So a dentist who named Colgate along with three other brands was counted as recommending Colgate.
00:07:51.920 --> 00:07:58.800
The ad made it sound like 80% of dentists chose Colgate over every other brand.
00:07:59.040 --> 00:08:00.959
The survey never showed that.
00:08:01.279 --> 00:08:05.680
Same people, a different question, a very different number.
00:08:05.839 --> 00:08:09.519
And Colgate was told not to use the claim again.
00:08:10.480 --> 00:08:14.959
So here's what we can ask of any number before we let it guide us.
00:08:15.199 --> 00:08:16.399
Who was counted?
00:08:16.639 --> 00:08:26.000
Men, women, children, older people, people with one health problem, or the complicated real life mix most of us carry.
00:08:26.319 --> 00:08:27.839
Who was left out?
00:08:28.160 --> 00:08:31.439
Sometimes on purpose, sometimes by accident?
00:08:31.920 --> 00:08:34.399
The phone directory in 1936?
00:08:34.799 --> 00:08:36.960
Women in early drug trials.
00:08:37.360 --> 00:08:41.519
Often older people and anyone with more than one health condition.
00:08:42.159 --> 00:08:43.919
Who chose to answer?
00:08:44.159 --> 00:08:50.159
People who mail back a ballot or fill out a survey can be very different from the people who don't.
00:08:50.399 --> 00:08:58.080
Think about who answers a survey about a supplement, often the people who loved it and the people who hated it.
00:08:58.320 --> 00:09:05.919
The ones in the middle rarely bother, and a survey that only hears from the delighted will always sound delightful.
00:09:06.399 --> 00:09:10.720
Who dropped out or was weeded out before the study began?
00:09:11.120 --> 00:09:21.440
If the people who had side effects quit halfway through and only the people who finished are counted, the treatment can look gentler than it really is.
00:09:21.919 --> 00:09:26.399
Always ask how many started and how many finished.
00:09:26.639 --> 00:09:29.200
And finally, the one that matters most.
00:09:29.600 --> 00:09:30.960
Are they like me?
00:09:31.440 --> 00:09:35.360
Now I want to be fair, good researchers know all of this.
00:09:35.600 --> 00:09:37.919
And when they count well, it shows.
00:09:38.240 --> 00:09:45.120
Gallup counted well in 1936, the women's health study counted women when it was finally done.
00:09:45.440 --> 00:09:50.720
Counting well means choosing people who look like the people the answer will be used on.
00:09:50.960 --> 00:09:54.960
It means saying plainly who was included and who wasn't.
00:09:55.120 --> 00:09:59.840
And it means not dressing up a narrow sample as if it spoke for everyone.
00:10:00.159 --> 00:10:09.039
So when you find a study that tells you exactly who was in it, men and women, young and old, healthy and not, that's a good sign.
00:10:09.200 --> 00:10:13.279
It's being honest with you about the limits of its answers.
00:10:13.600 --> 00:10:16.639
Why this brings me back to the person in front of me.
00:10:16.799 --> 00:10:21.679
This is why I care so much about observing the person who's actually in front of me.
00:10:21.919 --> 00:10:25.679
In my treatment room, no one gets left out of the sample.
00:10:25.919 --> 00:10:29.840
The woman with three conditions and five medications counts.
00:10:30.080 --> 00:10:32.320
The man in his 80s counts.
00:10:32.559 --> 00:10:37.200
The person whose body doesn't behave like the textbook counts.
00:10:37.519 --> 00:10:40.240
Studies give us averages from groups.
00:10:40.480 --> 00:10:44.159
Careful observation tells us about one real person.
00:10:44.399 --> 00:10:45.919
And we need both.
00:10:46.159 --> 00:10:55.919
But when the study never counted people like you, your own record and a practitioner who pays close attention may be the best evidence you have.
00:10:56.559 --> 00:11:03.039
Last week I invited you to look at your tongue and feel your pulse each morning and write down what you noticed.
00:11:03.840 --> 00:11:05.840
That's a sample of one.
00:11:06.000 --> 00:11:09.919
And it's the one sample that's guaranteed to include you.
00:11:10.159 --> 00:11:14.879
No one weeded you out, no one asked you the question in a tricky way.
00:11:15.120 --> 00:11:18.799
No one chose you because you owned a phone in 1936.
00:11:19.440 --> 00:11:21.919
It won't tell you what's true for everyone.
00:11:22.159 --> 00:11:23.519
That isn't its job.
00:11:23.759 --> 00:11:26.320
Its job is to tell you what's true for you.
00:11:26.559 --> 00:11:35.519
And when you bring that record into the room with a practitioner who knows how to read it, you've added the one person every study left out.
00:11:35.840 --> 00:11:40.480
So the next time we hear studies show, we can ask, in whom?
00:11:40.960 --> 00:11:51.360
The next time a dose is recommended, we can ask whether it was tested in bodies like ours, our sex, our age, our size, our other conditions.
00:11:52.000 --> 00:12:00.799
That can be as simple as asking, was this tested in people like me, women my age, people who take other medicines I take?
00:12:01.120 --> 00:12:04.399
A good practitioner won't be offended by that question.
00:12:04.639 --> 00:12:07.600
A good practitioner will be glad you asked.
00:12:07.759 --> 00:12:17.200
And the next time a number sounds overwhelming, 80%, 2 million answers, we can remember the literary digest.
00:12:17.600 --> 00:12:21.200
A big number can still be counting the wrong people.
00:12:22.080 --> 00:12:24.480
This week we look behind the number.
00:12:24.720 --> 00:12:26.399
Here's today's practice.
00:12:26.639 --> 00:12:29.519
Find one health claim that mentions a study.
00:12:29.759 --> 00:12:31.840
Then go looking for who was in it.
00:12:32.000 --> 00:12:38.639
You can often find it in the first few lines of a study summary or in a news story that describes it.
00:12:38.879 --> 00:12:45.600
Write down three things, their sex, their age, and what health conditions they had or didn't have.
00:12:45.840 --> 00:12:49.279
Then ask yourself honestly, are they like me?
00:12:49.600 --> 00:12:53.440
And if they're not, how much weight should I give the answer?
00:12:54.320 --> 00:13:03.279
Today we saw a poll of 2.4 million people miss an election by a landslide because it counted the wrong people.
00:13:03.519 --> 00:13:11.519
We saw a famous aspirin study count only men and a trial in women 16 years later find a different answer.
00:13:11.759 --> 00:13:18.960
We saw a sleeping pill dose the same for women and men until we learned their bodies handled it differently.
00:13:19.120 --> 00:13:25.919
And we saw 80% of dentists shrink once we learned how the question was really asked.
00:13:26.879 --> 00:13:29.039
And we saw why it matters.
00:13:29.360 --> 00:13:32.799
A number is only as good as the people behind it.
00:13:33.039 --> 00:13:38.240
When we know who was counted, we can decide whether the answer belongs to us.
00:13:38.639 --> 00:13:53.440
Tomorrow, we look at two of the slippery words in health news, the word average and the word risk, and how quote unquote cut your risk in half can mean one person out of a hundred.
00:13:53.679 --> 00:13:55.200
I'll meet you there.
00:13:56.399 --> 00:13:58.159
Keep the conversation going.
00:13:58.320 --> 00:14:03.039
Find your next steps in notes from a fellow traveler, my blog.
00:00:09.519 --> 00:00:12.720
Hello to all of my fellow travelers out there.
00:00:12.960 --> 00:00:14.560
I'm glad you're here.
00:00:14.880 --> 00:00:19.359
Wherever you're listening from, thank you for spending time with me today.
00:00:20.079 --> 00:00:22.000
Let's dive right in.
00:00:22.480 --> 00:00:26.480
Yesterday we saw that doubt can be manufactured.
00:00:26.719 --> 00:00:32.960
One industry wrote that doubt was its product, and we learned three questions to carry.
00:00:33.280 --> 00:00:34.880
Who is telling me this?
00:00:35.119 --> 00:00:36.479
Who paid for it?
00:00:36.719 --> 00:00:39.600
And what would they gain if I believed it?
00:00:39.920 --> 00:00:44.079
Today we asked the most basic question behind any number.
00:00:44.640 --> 00:00:46.399
Who was counted?
00:00:46.719 --> 00:00:55.920
In 1936, a popular magazine called the Literary Digest ran the biggest election poll America had ever seen.
00:00:56.159 --> 00:00:59.200
They mailed ballots to 10 million people.
00:00:59.759 --> 00:01:02.320
2.4 million sent them back.
00:01:02.880 --> 00:01:05.439
2.4 million answers.
00:01:05.680 --> 00:01:07.439
The result was clear.
00:01:07.760 --> 00:01:14.239
Elf Landon would beat Franklin Roosevelt with about 57% of the vote.
00:01:14.719 --> 00:01:21.920
Roosevelt won with about 62%, one of the biggest landslides in American history.
00:01:22.480 --> 00:01:27.120
How could 2.4 million answers be so wrong?
00:01:27.439 --> 00:01:38.000
The magazine had built its mailing list from phone directories, car registrations, and magazine subscriptions in the middle of the Great Depression.
00:01:38.319 --> 00:01:45.280
People who owned phones and cars and subscribed to magazines were not a fair picture of the country.
00:01:45.519 --> 00:01:49.599
And the people who chose to mail back a ballot weren't either.
00:01:49.840 --> 00:01:55.439
Meanwhile, a young pollster named George Gallup took a much smaller survey.
00:01:55.680 --> 00:02:03.840
Far fewer people, but he worked to choose a cross-section that looked like the country, and he called the election correctly.
00:02:04.159 --> 00:02:09.439
A huge number, the wrong people, a small number, the right people.
00:02:10.000 --> 00:02:13.120
That's today's lesson in two sentences.
00:02:13.360 --> 00:02:15.120
Why does this matter?
00:02:15.439 --> 00:02:20.000
Because every health claim we hear is based on some group of people.
00:02:20.319 --> 00:02:28.800
Every dose, every guideline, every study show, if that group looks like us, the answer may fit us.
00:02:29.039 --> 00:02:31.520
If it doesn't, the answer may not.
00:02:31.840 --> 00:02:36.000
And most of the time, no one tells us who was in the room.
00:02:36.319 --> 00:02:45.199
Many of us have had the experience of a treatment that was supposed to work and didn't, or a dose that was supposed to be right and wasn't.
00:02:45.439 --> 00:02:47.520
Sometimes the reason is simple.
00:02:47.840 --> 00:02:50.400
People like us were never counted.
00:02:50.800 --> 00:02:52.319
Here's where I stand.
00:02:52.639 --> 00:02:57.520
I believe medicine tends to be designed around whoever was counted.
00:02:57.680 --> 00:03:02.159
And for a long time, the people counted were a narrow slice of humanity.
00:03:02.400 --> 00:03:08.639
When you weren't in the study, you're being asked to trust an answer that was never tested on you.
00:03:09.120 --> 00:03:11.599
That isn't a reason to reject research.
00:03:11.759 --> 00:03:16.560
It's a reason to ask every single time who was in it.
00:03:16.879 --> 00:03:18.639
And I'll say something else.
00:03:18.960 --> 00:03:20.719
Size impresses us.
00:03:20.960 --> 00:03:23.199
A study of 20,000 people?
00:03:23.680 --> 00:03:25.280
Millions of users?
00:03:25.599 --> 00:03:29.199
But size never fixes a sample that's tilted.
00:03:29.759 --> 00:03:32.319
The literary digest proved that.
00:03:32.639 --> 00:03:36.960
More of the wrong people just makes the wrong answer louder.
00:03:37.360 --> 00:03:46.400
In 1981, researchers invited male doctors across the United States to join a study on aspirin and the heart.
00:03:46.719 --> 00:03:49.520
More than 22,000 men took part.
00:03:49.759 --> 00:03:52.319
It was called the Physician's Health Study.
00:03:52.560 --> 00:03:55.520
In 1989, the results came out.
00:03:55.840 --> 00:04:01.039
Men who took aspirin had 44% fewer first heart attacks.
00:04:01.280 --> 00:04:07.520
The finding was so strong that the aspirin part of the study was stopped years early.
00:04:07.759 --> 00:04:13.840
It was a landmark, and for years it shaped advice about aspirin for heart health.
00:04:14.159 --> 00:04:16.240
But notice who was counted?
00:04:16.720 --> 00:04:18.959
Men, only men.
00:04:19.519 --> 00:04:26.000
It took until 2005 for a large trial to answer the same question in women.
00:04:26.399 --> 00:04:34.319
The women's health study followed nearly 40,000 women over age 45 for about 10 years.
00:04:34.560 --> 00:04:38.720
Half took low dose aspirin and half took a placebo.
00:04:39.920 --> 00:04:44.480
For women, aspirin did not lower the risk of a first heart attack.
00:04:44.720 --> 00:04:49.759
The numbers were almost identical, about 1% in each group.
00:04:50.000 --> 00:04:57.120
It did lower the risk of stroke a little, but the heart attack benefit seen in men simply wasn't there.
00:04:57.600 --> 00:04:59.120
16 years.
00:04:59.439 --> 00:05:04.639
For 16 years, women were living with an answer that had been found in men.
00:05:05.199 --> 00:05:08.079
And notice one more thing about the men's study.
00:05:08.319 --> 00:05:14.639
Before the main trial began, more than 33,000 doctors started a trial period.
00:05:14.879 --> 00:05:20.000
Only about 22,000 went on to the rest of the real study.
00:05:20.240 --> 00:05:25.600
The trial period was a way of keeping the people most likely to stick with the pills.
00:05:25.839 --> 00:05:29.360
That's a common practice, and there are reasons for it.
00:05:29.600 --> 00:05:36.560
But it means the study counted people who had already shown they could take the pills and who stuck with them.
00:05:38.240 --> 00:05:44.560
When a sleeping pill met a woman's body, here's another story about who was counted.
00:05:44.879 --> 00:05:56.399
For years, a popular sleeping pill called Ambient or Zulpadem was prescribed at the same starting dose for men and women, 10 milligrams.
00:05:56.639 --> 00:06:06.720
Then in January of 2013, the FDA told the makers to cut the recommended dose for women in half, from 10 milligrams to 5.
00:06:07.279 --> 00:06:07.920
Why?
00:06:08.639 --> 00:06:12.560
Because women clear the drug from their bodies more slowly.
00:06:12.800 --> 00:06:21.759
That meant some women were waking up with enough of it still in their blood to impair them the next morning, especially behind the wheel of a car.
00:06:22.000 --> 00:06:29.040
Women had been taking it for about 20 years before the dose was cut in half for them.
00:06:29.279 --> 00:06:32.240
Think about what that means in a real life.
00:06:32.480 --> 00:06:37.519
A woman takes the dose her doctor prescribed exactly as prescribed.
00:06:37.680 --> 00:06:42.399
She sleeps, she gets up, makes breakfast, and drives her kids to school.
00:06:42.639 --> 00:06:48.160
She has no idea that her body is still carrying enough of the drug to slow her reactions.
00:06:48.480 --> 00:06:50.079
She did everything right.
00:06:50.319 --> 00:06:53.839
The dose was simply built on the wrong body.
00:06:55.279 --> 00:07:00.000
And it wasn't an accident that women were missing from so much early research.
00:07:00.240 --> 00:07:09.199
In 1977, the FDA issued guidance that kept women who could become pregnant out of the earliest stages of drug testing.
00:07:09.680 --> 00:07:13.360
That guidance wasn't changed until 1993.
00:07:14.480 --> 00:07:19.519
Sometimes the problem isn't who was counted, it's how they were counted.
00:07:19.759 --> 00:07:26.399
For years, Colgate ran ads saying that 80% of dentists recommend Colgate.
00:07:26.720 --> 00:07:29.680
80% sounds like a landslide.
00:07:29.920 --> 00:07:36.639
In 2007, the advertising watchdog in the United Kingdom ruled the claim misleading.
00:07:36.959 --> 00:07:37.600
Why?
00:07:37.839 --> 00:07:44.319
The survey had let dentists recommend several toothpaste brands, not just one.
00:07:45.040 --> 00:07:51.680
So a dentist who named Colgate along with three other brands was counted as recommending Colgate.
00:07:51.920 --> 00:07:58.800
The ad made it sound like 80% of dentists chose Colgate over every other brand.
00:07:59.040 --> 00:08:00.959
The survey never showed that.
00:08:01.279 --> 00:08:05.680
Same people, a different question, a very different number.
00:08:05.839 --> 00:08:09.519
And Colgate was told not to use the claim again.
00:08:10.480 --> 00:08:14.959
So here's what we can ask of any number before we let it guide us.
00:08:15.199 --> 00:08:16.399
Who was counted?
00:08:16.639 --> 00:08:26.000
Men, women, children, older people, people with one health problem, or the complicated real life mix most of us carry.
00:08:26.319 --> 00:08:27.839
Who was left out?
00:08:28.160 --> 00:08:31.439
Sometimes on purpose, sometimes by accident?
00:08:31.920 --> 00:08:34.399
The phone directory in 1936?
00:08:34.799 --> 00:08:36.960
Women in early drug trials.
00:08:37.360 --> 00:08:41.519
Often older people and anyone with more than one health condition.
00:08:42.159 --> 00:08:43.919
Who chose to answer?
00:08:44.159 --> 00:08:50.159
People who mail back a ballot or fill out a survey can be very different from the people who don't.
00:08:50.399 --> 00:08:58.080
Think about who answers a survey about a supplement, often the people who loved it and the people who hated it.
00:08:58.320 --> 00:09:05.919
The ones in the middle rarely bother, and a survey that only hears from the delighted will always sound delightful.
00:09:06.399 --> 00:09:10.720
Who dropped out or was weeded out before the study began?
00:09:11.120 --> 00:09:21.440
If the people who had side effects quit halfway through and only the people who finished are counted, the treatment can look gentler than it really is.
00:09:21.919 --> 00:09:26.399
Always ask how many started and how many finished.
00:09:26.639 --> 00:09:29.200
And finally, the one that matters most.
00:09:29.600 --> 00:09:30.960
Are they like me?
00:09:31.440 --> 00:09:35.360
Now I want to be fair, good researchers know all of this.
00:09:35.600 --> 00:09:37.919
And when they count well, it shows.
00:09:38.240 --> 00:09:45.120
Gallup counted well in 1936, the women's health study counted women when it was finally done.
00:09:45.440 --> 00:09:50.720
Counting well means choosing people who look like the people the answer will be used on.
00:09:50.960 --> 00:09:54.960
It means saying plainly who was included and who wasn't.
00:09:55.120 --> 00:09:59.840
And it means not dressing up a narrow sample as if it spoke for everyone.
00:10:00.159 --> 00:10:09.039
So when you find a study that tells you exactly who was in it, men and women, young and old, healthy and not, that's a good sign.
00:10:09.200 --> 00:10:13.279
It's being honest with you about the limits of its answers.
00:10:13.600 --> 00:10:16.639
Why this brings me back to the person in front of me.
00:10:16.799 --> 00:10:21.679
This is why I care so much about observing the person who's actually in front of me.
00:10:21.919 --> 00:10:25.679
In my treatment room, no one gets left out of the sample.
00:10:25.919 --> 00:10:29.840
The woman with three conditions and five medications counts.
00:10:30.080 --> 00:10:32.320
The man in his 80s counts.
00:10:32.559 --> 00:10:37.200
The person whose body doesn't behave like the textbook counts.
00:10:37.519 --> 00:10:40.240
Studies give us averages from groups.
00:10:40.480 --> 00:10:44.159
Careful observation tells us about one real person.
00:10:44.399 --> 00:10:45.919
And we need both.
00:10:46.159 --> 00:10:55.919
But when the study never counted people like you, your own record and a practitioner who pays close attention may be the best evidence you have.
00:10:56.559 --> 00:11:03.039
Last week I invited you to look at your tongue and feel your pulse each morning and write down what you noticed.
00:11:03.840 --> 00:11:05.840
That's a sample of one.
00:11:06.000 --> 00:11:09.919
And it's the one sample that's guaranteed to include you.
00:11:10.159 --> 00:11:14.879
No one weeded you out, no one asked you the question in a tricky way.
00:11:15.120 --> 00:11:18.799
No one chose you because you owned a phone in 1936.
00:11:19.440 --> 00:11:21.919
It won't tell you what's true for everyone.
00:11:22.159 --> 00:11:23.519
That isn't its job.
00:11:23.759 --> 00:11:26.320
Its job is to tell you what's true for you.
00:11:26.559 --> 00:11:35.519
And when you bring that record into the room with a practitioner who knows how to read it, you've added the one person every study left out.
00:11:35.840 --> 00:11:40.480
So the next time we hear studies show, we can ask, in whom?
00:11:40.960 --> 00:11:51.360
The next time a dose is recommended, we can ask whether it was tested in bodies like ours, our sex, our age, our size, our other conditions.
00:11:52.000 --> 00:12:00.799
That can be as simple as asking, was this tested in people like me, women my age, people who take other medicines I take?
00:12:01.120 --> 00:12:04.399
A good practitioner won't be offended by that question.
00:12:04.639 --> 00:12:07.600
A good practitioner will be glad you asked.
00:12:07.759 --> 00:12:17.200
And the next time a number sounds overwhelming, 80%, 2 million answers, we can remember the literary digest.
00:12:17.600 --> 00:12:21.200
A big number can still be counting the wrong people.
00:12:22.080 --> 00:12:24.480
This week we look behind the number.
00:12:24.720 --> 00:12:26.399
Here's today's practice.
00:12:26.639 --> 00:12:29.519
Find one health claim that mentions a study.
00:12:29.759 --> 00:12:31.840
Then go looking for who was in it.
00:12:32.000 --> 00:12:38.639
You can often find it in the first few lines of a study summary or in a news story that describes it.
00:12:38.879 --> 00:12:45.600
Write down three things, their sex, their age, and what health conditions they had or didn't have.
00:12:45.840 --> 00:12:49.279
Then ask yourself honestly, are they like me?
00:12:49.600 --> 00:12:53.440
And if they're not, how much weight should I give the answer?
00:12:54.320 --> 00:13:03.279
Today we saw a poll of 2.4 million people miss an election by a landslide because it counted the wrong people.
00:13:03.519 --> 00:13:11.519
We saw a famous aspirin study count only men and a trial in women 16 years later find a different answer.
00:13:11.759 --> 00:13:18.960
We saw a sleeping pill dose the same for women and men until we learned their bodies handled it differently.
00:13:19.120 --> 00:13:25.919
And we saw 80% of dentists shrink once we learned how the question was really asked.
00:13:26.879 --> 00:13:29.039
And we saw why it matters.
00:13:29.360 --> 00:13:32.799
A number is only as good as the people behind it.
00:13:33.039 --> 00:13:38.240
When we know who was counted, we can decide whether the answer belongs to us.
00:13:38.639 --> 00:13:53.440
Tomorrow, we look at two of the slippery words in health news, the word average and the word risk, and how quote unquote cut your risk in half can mean one person out of a hundred.
00:13:53.679 --> 00:13:55.200
I'll meet you there.
00:13:56.399 --> 00:13:58.159
Keep the conversation going.
00:13:58.320 --> 00:14:03.039
Find your next steps in notes from a fellow traveler, my blog.
