Full Transcript for Selection bias in cluster randomised controlled trials

in the sense that said there are a number that's in advance and then put them the younger patients. 49 00:05:12,040 --> 00:05:18,130 So 75, for example, was put in all men under the age of 40 in the intervention group, 50 00:05:18,280 --> 00:05:23,080 an old men over the age of 40 and the control group and says, I'm completely biased. 51 00:05:25,920 --> 00:05:29,280 A colleague of mine who did a study in this area often hear it. 52 00:05:29,610 --> 00:05:41,820 What she did was took all the Vanderbilt controlled trials published in one year in the main medical journal, the medical journal from jail, etc., 53 00:05:43,050 --> 00:05:52,800 and clocked them inadequate or adequate and she inadequate concealment of even opaque envelopes 99% of file she classed as being 54 00:05:52,800 --> 00:06:00,030 inadequate all use of fake field envelopes adequate ones with it used a web based or telephone randomisation was difficult, 55 00:06:00,240 --> 00:06:05,760 and she basically found that the p value for those files that envelopes that 56 00:06:05,760 --> 00:06:13,610 2.0 to compare just got over 0.05 for those that will use adequate charts. 57 00:06:13,620 --> 00:06:18,360 So this is the plot of the study. You can see the. 58 00:06:23,670 --> 00:06:35,950 So if you can see the adequate ones, the dark line that can see and this is about 5%, you see about half the trials are statistically significant, 59 00:06:35,970 --> 00:06:41,760 half and not just about what you would expect if you but if you look at the inadequate one six, 60 00:06:41,820 --> 00:06:53,340 the orange you like see more trials that are that are statistically significant compared with the adequately concealed one. 61 00:06:53,940 --> 00:06:57,490 Now, the only reasonable assumption is pattern here for people. 62 00:06:57,570 --> 00:07:01,920 The envelopes have been opened in advance or people are filling the allocation 63 00:07:02,130 --> 00:07:06,060 because I can't think of any reason why people who choose to use sealed, 64 00:07:06,060 --> 00:07:15,960 opaque envelopes to conceal the allocation are able to pick interventions of more likely to succeed than people who use the more secure method. 65 00:07:16,230 --> 00:07:22,020 It doesn't make sense. This is a masterful design. 66 00:07:22,200 --> 00:07:35,370 What she did was took a dozen systematic view of randomised controlled trials that all published in BMJ Jarman and 2012 and said. 67 00:07:35,970 --> 00:07:40,700 And then what she did was extracted the data from the component trials, the systematic review. 68 00:07:40,710 --> 00:07:49,290 So the first one she got the ten papers and then what she did was that she was a matter analysis of age between the two groups, 69 00:07:49,620 --> 00:07:54,899 both with age should be what you should get within a mention of age. 70 00:07:54,900 --> 00:08:00,360 In randomised controlled trial you should get zero heterogeneity because the knowledge we know, 71 00:08:00,720 --> 00:08:04,890 we know the difference between groups is true, which is no age difference through randomisation. 72 00:08:05,250 --> 00:08:09,820 Those should get no heterogeneity and also we should get no difference. 73 00:08:10,290 --> 00:08:17,999 So if you look at the first one, that's the narrative by the IWF there is enormous a 4% to little difference and 74 00:08:18,000 --> 00:08:23,430 I think it's different eight that's less important than the heterogeneity. 75 00:08:23,550 --> 00:08:32,830 In fact if you look at have that for the study that for hideous and it's the only four for trials 76 00:08:32,940 --> 00:08:37,810 showed zero hitches in that rough full meta analysis that have shown selection narrative. 77 00:08:38,070 --> 00:08:43,920 So all those match analysis contain trials but the allocations have been subverted as affecting baseline. 78 00:08:44,430 --> 00:08:53,700 And even if those for one of them the age imbalance at this particular trial that is populated by a host of the trial. 79 00:08:56,260 --> 00:09:00,340 So yeah, that's a very big problem. 80 00:09:01,450 --> 00:09:05,710 So the problem is individually randomised trials is really quite widespread. 81 00:09:05,830 --> 00:09:13,569 It's damaging metal analysis. See, this is a trial where well I don't know about you guys, 82 00:09:13,570 --> 00:09:24,250 but I've read my medical journal with trials and then mainly just to sort of pick it out if taking one to say this is what this was one along. 83 00:09:24,550 --> 00:09:29,990 Yeah, this is a block randomised 54 patient fleet block with seven randomisation centres, 84 00:09:30,340 --> 00:09:35,950 blocks of four cartons of duty, content to cars marked with house officer, all with nurse. 85 00:09:36,130 --> 00:09:42,100 Each card is placed into a envelope and sealed. The block was shuffled and after shuffling the block faces in a box. 86 00:09:42,400 --> 00:09:53,950 So this is a trial with which problem? That way if you don't get it because the statistician on this job didn't observe it either. 87 00:09:54,400 --> 00:10:01,180 The editor of the Journal and the BMJ didn't sit and neither did the statistical editor didn't see it all. 88 00:10:01,570 --> 00:10:08,710 That is blindingly bloody obvious when when you actually block randomisation block for stuff side by side. 89 00:10:09,070 --> 00:10:15,240 So the imbalance can only be half the block size because we've got dodgy business going on. 90 00:10:15,250 --> 00:10:18,450 Southampton V Sheffield how much we can't. 91 00:10:18,460 --> 00:10:22,930 It's impossible and arithmetically impossible for that to happen naturally either. 92 00:10:23,020 --> 00:10:27,130 The lead author who ignored me then I've got to stop fishing on the paper as well on basis of 93 00:10:27,280 --> 00:10:31,540 the lead author sometimes ignores it because I think if some of the statistics are ignored, 94 00:10:31,540 --> 00:10:34,640 if I wrote fascinating, ordinary,

well, so on. 95 00:10:35,300 --> 00:10:42,250 So because there might be a reasonable explanation for I couldn't understand what was a reasonable explanation that I got long time ago. 96 00:10:42,250 --> 00:10:46,480 This just happened just as the largest trial in this area. 97 00:10:47,830 --> 00:10:52,940 That lesson I'm a bit of an internal medicine because [INAUDIBLE] have never published anything about attention, 98 00:10:53,530 --> 00:10:57,250 but they should publish this complete twaddle. So what is it? 99 00:10:58,510 --> 00:11:10,450 So we randomised 450 women and over 900 women to undergraduate students to an intervention to try and prevent them. 100 00:11:11,200 --> 00:11:18,010 Would you say they still haven't been sexually assaulted in the first year at university and the intervention clearly appears it worked. 101 00:11:18,340 --> 00:11:26,800 So in the intervention group 5% were raped compared to 10% in the control group. 102 00:11:27,040 --> 00:11:40,269 So halving of the intervention. And then so what's the problem you saw in randomisation was performing 70 blocks of two with the use of online tool 103 00:11:40,270 --> 00:11:49,360 randomisation that was suffocation code to sign for the 52% control group of 60 for the Prevention Resistance Group. 104 00:11:50,770 --> 00:11:55,900 See, the problem then were if you don't, the statistician could fit you in the jail. 105 00:11:57,280 --> 00:12:03,790 They got you've got, what is it three sites block size of two stuff side by side. 106 00:12:04,120 --> 00:12:04,899 The intervention. 107 00:12:04,900 --> 00:12:11,620 You can only be in balance by half block size, which is why they can only possibly gain balance across the whole trial by three participants. 108 00:12:13,330 --> 00:12:16,299 I mean, I've got a level math that surely, you know. That's right. 109 00:12:16,300 --> 00:12:24,160 And that the site found to have half of two is one, three times one, two, three anyway. 110 00:12:24,370 --> 00:12:28,860 So 450 to respond to the control group and 464 to resistance. 111 00:12:29,290 --> 00:12:32,410 So that isn't to me it's what is it, 12. 112 00:12:32,860 --> 00:12:36,010 Anyway, I did write to actually take it back. 113 00:12:36,140 --> 00:12:40,030 You're concerned about half of the students on the publish my lecture about this trial. 114 00:12:41,200 --> 00:12:48,759 And so given that you about that they did a good publisher and the authors responded 115 00:12:48,760 --> 00:12:57,140 by saying that there were 10 to 1 device in error because they had people misspell, 116 00:12:57,160 --> 00:13:00,489 you know. So irritatingly, my name gets me stuff quite a lot. 117 00:13:00,490 --> 00:13:05,230 People from the middle are in the main court with towards the Centre for Progress which donated it. 118 00:13:07,390 --> 00:13:11,740 And so you can imagine if somebody was told to send in the name, somebody would have written it down, 119 00:13:11,750 --> 00:13:16,389 told and someone else would have written that order and then they thought they are two different people and randomise. 120 00:13:16,390 --> 00:13:19,990 And quite a funny that all ten ended up in one group. 121 00:13:19,990 --> 00:13:26,360 I don't know what your probability of of that happen is pretty remote but anyway that passed. 122 00:13:27,670 --> 00:13:41,150 So we know and what is run by for a few portion of the person because they were using small block sizes and and envelope. 123 00:13:41,560 --> 00:13:45,040 And so what about cluster randomisation now. 124 00:13:45,520 --> 00:13:54,759 Cluster randomisation just to remind everybody what the first got to unit the unit of allocation is something like a hospital or GP practice. 125 00:13:54,760 --> 00:13:56,590 It might be stored, it might be brought, 126 00:13:56,680 --> 00:14:06,820 the kind of people in back then you've got the individuals who are associated with that get allocated to the treatment or not. 127 00:14:07,600 --> 00:14:10,580 You have to run lots of stuff to do that. 128 00:14:11,050 --> 00:14:19,390 And then the unit of specimen, the outcome assessment is on the on the usually on the patient, the individual patient at the psychiatric cluster. 129 00:14:20,650 --> 00:14:27,070 And so we've got two elements. We've got potential bias and the actual randomisation to the clusters, 130 00:14:27,070 --> 00:14:33,070 which is the same as that can happen for individually randomised controlled trial then. 131 00:14:33,820 --> 00:14:37,310 But we can sort that out that they look out for. 132 00:14:37,330 --> 00:14:44,230 The next problem of course, which is that tractable is what happens if you have to recruit the participants after you randomise. 133 00:14:45,640 --> 00:14:57,000 So. Selection bias can be randomised to clusters so long as you've done that properly by an independent person. 134 00:14:57,570 --> 00:15:03,510 You've got balance about the quality of that. What were all happened to that then? 135 00:15:03,510 --> 00:15:12,030 What really happened? How the selection occurs is when they start recruiting the patients actually randomise because 136 00:15:12,030 --> 00:15:16,380 just as individual randomisation is typically because people know the allocation in advance. 137 00:15:16,740 --> 00:15:22,229 So it's just that because you have two people who know the allocation back to the clinician, 138 00:15:22,230 --> 00:15:31,770 those allocation who can then select patients to go in for some reason or, or you can have the patient and the patient that is the across allocation. 139 00:15:31,770 --> 00:15:36,030 So they can volunteer to go in the study directly between the physician. 140 00:15:37,080 --> 00:15:45,150 So you get clustered. But the first time I sort of tweaked this was back in the nineties when I first met some 141 00:15:45,240 --> 00:15:52,440 of these people with the health problems on a lot of back been beans and we would

have 142 00:15:52,440 --> 00:16:01,200 done a pilot very complicated design to my ex boss for my being had followed me around 143 00:16:01,200 --> 00:16:06,909 New York so I was working with him again and so he loved having very complicated design. 144 00:16:06,910 --> 00:16:12,630 And this was that the initial design for this trial was a cluster randomised flip flop design. 145 00:16:14,040 --> 00:16:22,320 And so we did a pilot before we launched the main project check the cockpit design work. 146 00:16:22,980 --> 00:16:30,200 So we ran the so to show you what happened at those 26 practices with some kind of 147 00:16:30,210 --> 00:16:39,780 stratification to randomisation bums on the number of the patients in the intervention group. 148 00:16:39,780 --> 00:16:48,720 Then you see training that was management so that GP's and practice managers would receive some training on how to manage patients with low back pain. 149 00:16:49,350 --> 00:16:59,390 Basically try and get them to be mobilised as much as possible because lying in bed, much more chronic, like with make it chronic to train those. 150 00:16:59,400 --> 00:17:04,740 And it's almost obvious in hindsight what would happen if you're telling me about back pain. 151 00:17:04,950 --> 00:17:08,340 I didn't find patients back and hey presto, that's what they do. 152 00:17:08,520 --> 00:17:13,800 They identify a lot more patients, the usual category for that problem than usual practice. 153 00:17:14,070 --> 00:17:21,150 So by recruiting more than 2 to 3 times more patients than the control group. 154 00:17:21,420 --> 00:17:26,729 And then when you look at the baseline values like Rutland Life Disability School, 155 00:17:26,730 --> 00:17:35,370 Aberdeen Back Pain Scout and FFO six can see, quote, higher scores of those two is worth and the low score is worth that. 156 00:17:35,370 --> 00:17:39,929 So you can see the outcome. The main outcome would be the vote in my score. 157 00:17:39,930 --> 00:17:46,739 So the patient team, before they have a chance to get any kind of advice and treatment that the fact this was a significant work. 158 00:17:46,740 --> 00:17:54,120 So we have to abandon that trial because it was it was helpful. 159 00:17:54,120 --> 00:17:58,800 So fortunately, we've done a pilot, so we just then moved on without the cluster element to the study. 160 00:18:01,230 --> 00:18:13,920 This is the recruitment graph that we saw for the intervention for flu and we can see the old way through was like a double number of recruitment. 161 00:18:14,430 --> 00:18:18,509 The good the only good thing about this pilot study is it shows that if you teach, 162 00:18:18,510 --> 00:18:23,069 if you train GP's about a clinical condition, they will recruit more for you. 163 00:18:23,070 --> 00:18:29,640 So if you want to do it, you do it in a randomised trial in primary care we want to enhance recruitment training and they will recruit a lot more. 164 00:18:32,190 --> 00:18:42,300 So I wrote a paper about partly about this and suggesting that we should avoid cluster randomisation at all costs. 165 00:18:42,660 --> 00:18:53,380 And so what I recommend is that is that we should try and do individual randomisation for possible and out of 166 00:18:53,670 --> 00:19:01,559 these if you've got some contamination and I feel hold ups and if you've got contamination that goes up to 30%, 167 00:19:01,560 --> 00:19:06,720 it's still more cost cost effective to do the individual randomisation cluster. 168 00:19:07,920 --> 00:19:08,460 Okay. 169 00:19:08,940 --> 00:19:22,140 So in 2002, I inherited the trials in York and one of the trials had been done at that time, remember too from before for the cluster run Photoshop. 170 00:19:22,470 --> 00:19:31,680 And what we want to work is shoulder pain and they we're going to recruit patients who showed randomised 171 00:19:31,680 --> 00:19:38,280 the general practice trained GP's and then recruit the patients as a group of patients shoulder pain. 172 00:19:38,280 --> 00:19:41,670 And then we would look at the outcome of training on the GP, 173 00:19:43,020 --> 00:19:48,600 on the patients outcomes to see if it got better off patients, GP to being trained to deal with. 174 00:19:49,250 --> 00:19:56,209 Shoulder problem. And so when I joined it, I said, we really need to change the design, 175 00:19:56,210 --> 00:20:02,480 that the design and in the first place, we need to change the design here because it will be catastrophic. 176 00:20:03,890 --> 00:20:13,220 So when I go and if it were so chopped off the place up there, Japan obviously would have to contend with the several of them. 177 00:20:14,120 --> 00:20:18,480 And I couldn't persuade anybody else that we needed to change tack here. 178 00:20:19,790 --> 00:20:28,610 So we had a first draft committee, steering group steering committee statement of the owners. 179 00:20:29,150 --> 00:20:38,110 And so at this point, we could see. We have doubled in the chain saying GP so we could double the numbers of patients. 180 00:20:39,680 --> 00:20:46,630 So at this point I said, look, I know you thought a lot is wrong. 181 00:20:47,500 --> 00:20:53,139 I don't know. I think psychology helps with that because I think you shouldn't have it. 182 00:20:53,140 --> 00:20:56,860 If somebody is doing something, it's catastrophic and you've warned them against it. 183 00:20:56,860 --> 00:21:00,100 And I think it is probably problematic to tell them, I've told you so. 184 00:21:00,310 --> 00:21:04,100 You're supposed to set you know, you're not supposed to cry over the. 185 00:21:04,960 --> 00:21:08,230 But I think I can do that. I've heard that. 186 00:21:08,530 --> 00:21:14,560 Yes, you've got it wrong. We don't know. I may have had an external still. 187 00:21:15,160 --> 00:21:21,580 Have a quick look. The person from the audience they're not have external statistician that who said 188 00:21:22,180 --> 00:21:26,440 we

clinch an argument with that difference isn't statistically significant. 189 00:21:28,480 --> 00:21:36,370 My account for that was I told the story of the jumbo jet that flew over Indonesia in mid seventies when 190 00:21:36,370 --> 00:21:43,240 the volcano was going off and it flew it flew through the volcanic ash and all four engines will come out. 191 00:21:44,320 --> 00:21:49,270 So this is the analogy that you're positing said aeroplane 50,000 feet, 192 00:21:49,270 --> 00:21:54,819 all the engines have shut down and the co-pilot comes on and says, we've had a little trouble. 193 00:21:54,820 --> 00:21:58,299 All our engines have lost power, but don't worry, we've lost a thousand people. 194 00:21:58,300 --> 00:22:05,980 That's not an awfully significant. My view, if I was a passenger, is in about half an hour, an hour, it's going to become bloody. 195 00:22:05,980 --> 00:22:10,300 I have no idea that somebody has won it. At least one of the engines restarted. 196 00:22:10,660 --> 00:22:13,730 Fortunately, they got three of them off in one state. 197 00:22:13,750 --> 00:22:29,360 But I would say that it's not that it may not be statistically significant, but it's going to be if we carry on and we catch up not to the authentic. 198 00:22:30,040 --> 00:22:35,769 I told you so then I didn't tell you that at this point it was too late. 199 00:22:35,770 --> 00:22:39,579 Now we are discussing. 200 00:22:39,580 --> 00:22:43,690 So of course that's what's going on. 201 00:22:44,080 --> 00:22:48,820 We need more evidence. You go on about you think that is probably a one off. 202 00:22:49,030 --> 00:22:54,040 So a group was got to try and prove that this is a major problem. 203 00:22:54,340 --> 00:23:08,800 So we did a systematic review of well first published in the BMJ last year in the Journal, met us for five years from January 7th, October 2002. 204 00:23:09,460 --> 00:23:14,410 And so what you find we found that on the first cluster trial, 205 00:23:15,190 --> 00:23:24,220 about 40% was showing evidence of selection bias would tell by imbalances in their recruitment values or in covariate imbalances, 206 00:23:24,830 --> 00:23:29,650 all based on covaxin policy. And in fact, 207 00:23:29,860 --> 00:23:37,030 that's an underestimate because we came across another trial that published a secondary paper after they published a paper in 208 00:23:37,030 --> 00:23:45,850 The Lancet after was published that demonstrated data and admitted they had a selection bias of 40% actually in the letter. 209 00:23:46,660 --> 00:23:58,660 But you know, both those data a long, long time ago, so they do things differently depending on who said this happened. 210 00:23:59,170 --> 00:24:09,520 Unfortunately not so again, when doctors flipped the switch published in 2008, another that and where are we. 211 00:24:09,550 --> 00:24:17,300 Yes about 35 for trial that then showed recruitment bias is possible at five. 212 00:24:17,320 --> 00:24:26,440

Does they have clear evidence in that in the paper that there was recruitment by more than 213 00:24:27,250 --> 00:24:38,229 a thousand to the more recent cluster trials and she found huge problems of the 23 trial. 214 00:24:38,230 --> 00:24:43,090 She looked at about 48% of a high risk of selection bias. 215 00:24:44,740 --> 00:24:50,500 Now, something else she did was what I showed you before an individual randomised trial. 216 00:24:50,860 --> 00:24:56,200 She did looked at the heterogeneity again. 217 00:24:56,200 --> 00:25:09,130 She extracted all the baseline age of all of the 23 cluster trial and then looked to see if the baseline imbalances and what heterogeneity was said. 218 00:25:09,520 --> 00:25:15,130 We have one, two, three, four, five, six, 219 00:25:15,370 --> 00:25:21,639 seven plus the graph where the statistically significant imbalance in eight and look at 220 00:25:21,640 --> 00:25:29,890 hatch American not 3% now remember these are corrected to remember we're looking at. 221 00:25:34,120 --> 00:25:38,530 Limber up the knowledge to hear. 222 00:25:38,680 --> 00:25:45,560 This should be no different. They should all be. The differences make an individual trial that physically fit. 223 00:25:46,240 --> 00:25:50,770 And may even be statistically significant in my chance. You wouldn't expect this. 224 00:25:51,160 --> 00:25:55,059 As I said, this is what you should be fed. 225 00:25:55,060 --> 00:26:06,760 This is the individuals and the lifestyle that are not even myself taking the stage at them for the same issues. 226 00:26:08,800 --> 00:26:14,090 So to put it ten time, this is what we've got to do a better analysis of individual randomised trials. 227 00:26:15,670 --> 00:26:19,290 This is what you expect to say i squared 0%. 228 00:26:20,170 --> 00:26:23,260 We have a tiny, tiny difference which is nowhere near. 229 00:26:24,370 --> 00:26:29,260 We have two trials that have significant imbalances. 230 00:26:29,290 --> 00:26:33,890 No one goes a long way, really goes with what? Which is about what to expect. 231 00:26:34,780 --> 00:26:37,149 So this is what this is the difference. 232 00:26:37,150 --> 00:26:50,680 We would hope to see if we'd done the task, but we're not doing this trial with individually randomised trial yet and we have an appalling situation. 233 00:26:51,670 --> 00:27:00,940 I'll go through a quote, another case study. This is published a couple of years ago and got a lot of it, got a lot of press attention. 234 00:27:02,440 --> 00:27:09,130 I was able to use scientific contact information for Common Output published 235 00:27:09,580 --> 00:27:16,540 and I gave the new scientist a journalist saying how awful it was to publish. 236 00:27:16,540 --> 00:27:18,040 Unfortunately, she published. 237 00:27:18,160 --> 00:27:25,890 She gave a synopsis of the trial and said there was some controversy about it, but the conference is just me ranting to an hour ago. 238 00:27:27,370 --> 00:27:33,850 So this is I mean, this is to stop trying to stop teenage girls getting pregnant. 239 00:27:34,330 --> 00:27:45,879 So the of the randomised schools to have identified about 50 and they get randomised to get a baby simulator. 240 00:27:45,880 --> 00:27:54,050 So the girl has the weekend with a baby that cries at regular intervals, keeps up all night, nothing changed and fed all the cows. 241 00:27:54,050 --> 00:27:58,360 How anybody would have a baby knows exactly what it sounds like. 242 00:27:58,750 --> 00:28:08,860 And so the heart of all of this is supposed to put the the the mothers are the potential as it's the young women of getting himself pregnant. 243 00:28:10,270 --> 00:28:17,890 Now what is so statistically significant, different half of like 1.5, 244 00:28:18,100 --> 00:28:27,060 but it actually shows that the risk of getting pregnant in the intervention doubled the risk of the control of the birth. 245 00:28:27,420 --> 00:28:42,290 Then what we got had a baby compared to 4% control group and they did all lots of fancy statistics, very healthy babies, about half the p value. 246 00:28:42,880 --> 00:28:46,990 But the difference is, I don't know what a block binomial model is for anyway. 247 00:28:46,990 --> 00:28:53,260 We do that for all that us or looking at high quality data. 248 00:28:53,620 --> 00:28:57,130 But do we believe it? Don't believe a word of it. 249 00:28:57,280 --> 00:29:03,970 It's just, you know, statistics are meaningless if you've got awful data looking over data. 250 00:29:04,360 --> 00:29:16,570 This is selection bias on a massive scale. What the NHS thinks, and I think it was a baby doll simulation might actually increase pregnancy rates. 251 00:29:16,990 --> 00:29:18,430 They've gone to NHS choices. 252 00:29:18,430 --> 00:29:27,550 You'll find that this sort of study design with some sort of statisticians, the audience won't go to that paper, do some calculation. 253 00:29:27,760 --> 00:29:32,950 We've got that wrong, actually, but that's a minor problem. We've got the sample. So I've talked to a lot of it. 254 00:29:33,040 --> 00:29:40,240 I tend to look at sample size calculations very easy as why I do, but I'm not going to pay attention quickly. 255 00:29:40,390 --> 00:29:44,590 Do them. Because if the sample size, if you can't get sample size calculations wrong, which is right, 256 00:29:44,590 --> 00:29:49,200 which is a piece of cake, you can do it and you have to write just about a tiny bit. 257 00:29:50,290 --> 00:29:55,850 But then if they don't get that right, you know, then we get anything else like that. 258 00:29:57,250 --> 00:30:02,680 Okay. But this is a good study design on a sample size. 259 00:30:02,680 --> 00:30:12,399 So there we have it. Thank you. So I went to I decided to look for higher soul and more thought of some sort of evidence base. 260 00:30:12,400 --> 00:30:22,140 And so I naturally take that email for the Daily Mail Lifelike Baby Doll

designed in 30 days, just from having children actually very pregnant today, 261 00:30:23,050 --> 00:30:29,800 quite intensive search of size, attention to details when they were looking after the dolls, encourage them to have a baby. 262 00:30:30,760 --> 00:30:34,300 No, it doesn't cause a trial to say anything of the kind. 263 00:30:34,630 --> 00:30:37,900 But is it true? Well, 264 00:30:39,490 --> 00:30:44,530 it must be true because NHS choices on the Daily Mail scientists and they both say 265 00:30:44,530 --> 00:30:47,770 that giving infants simulated teenage girls increased their risk of pregnancy. 266 00:30:48,610 --> 00:30:51,610 I like to I have no shares in infant simulators. 267 00:30:53,500 --> 00:31:04,130 I think that probably don't work. But I certainly this shows that I don't have shares in that and I don't care one way or another about them except. 268 00:31:04,660 --> 00:31:07,690 Well, I mean, I suppose in some way I'll stop wasting money on them. 269 00:31:09,370 --> 00:31:13,630 We think let's have a look at well, I won't get into the sample size so that you do that yourself. 270 00:31:14,620 --> 00:31:24,549 This is what. So that's a bit of console time. So they want to buy 57 pills, put it on control to protect themselves. 271 00:31:24,550 --> 00:31:33,820 Right now, first thing we look at is the eligible students, because the proportion of the population in all these girls, 272 00:31:34,450 --> 00:31:40,480 3150 girls compared to have one of them girls that is you balanced population. 273 00:31:40,900 --> 00:31:47,080 If you if you include all those girls and intention st analysis you get them but estimate the truth 274 00:31:48,100 --> 00:31:54,220 so they didn't do that there's enough to go to you want to take part but only half equivalent. 275 00:31:55,240 --> 00:32:06,190 So even if they even if you've got the same something proportion in both total imagine doing it now imagine trying to publish an individual 276 00:32:06,190 --> 00:32:19,510 randomised control trust where 50% of your of your randomised samples are locked out and then you have 58% in the in the intervention group. 277 00:32:19,630 --> 00:32:24,040 But we have an 8% difference of quite a large difference in the number of girls in two groups. 278 00:32:24,400 --> 00:32:36,250 So this eight that immediately introduces selection but over and above any selection bias and then observable characteristics both. 279 00:32:38,720 --> 00:32:46,280 Based on that, you probably taught an office so I teach my students don't have a baseline character 280 00:32:46,780 --> 00:32:52,700 that baseline comparison to the main test what's the argument but there's not that. 281 00:32:53,210 --> 00:32:57,280 Well, we know the knowledge. So if you get the gist, the significant difference difference. 282 00:32:58,460 --> 00:33:06,160 So I'm changing my mind on the type of work and I feel like, oh, all the time about how we should pivot. 283 00:33:06,170 --> 00:33:09,980 The pattern based on variables are not the reason most people do it. 284 00:33:10,150 --> 00:33:16,220 Fact randomisation is work to look for selection, but I'm slowly gravitating to that. 285 00:33:16,460 --> 00:33:23,360 This is in The Lancet. So we've got our P values, but I've got my friendly statistician to my let's have a look. 286 00:33:23,900 --> 00:33:31,990 But this is trying to look at what type of main students what the numbers proportion of difference is huge the p values of scale. 287 00:33:32,270 --> 00:33:40,309 If we look at this now the these are teenage girls getting pregnant so we can expect socio 288 00:33:40,310 --> 00:33:45,710 demographic characteristics to be a high predictor of chances of getting pregnant as a teenager. 289 00:33:46,040 --> 00:33:51,350 And if we look at the factors, demographic P values, got it while we look at it. 290 00:33:51,650 --> 00:33:56,600 So we look at the lowest proportion of socioeconomic factors. 291 00:33:57,110 --> 00:34:02,059 We have 13 5%. 292 00:34:02,060 --> 00:34:11,150 So that in the intervention group, 13% of the girls are from the lowest 10% of socioeconomic status, 293 00:34:11,150 --> 00:34:27,049 compared with only 5% of the of the control group that we look at 17 now an alpha somebody and so you might 294 00:34:27,050 --> 00:34:34,730 do a propensity score matching you might or whatever stuff to try and control for observable imbalances. 295 00:34:34,850 --> 00:34:39,079 But I can bet there's a caveat in that that is not measured, 296 00:34:39,080 --> 00:34:44,930 but not that you are not controlling for in your analysis that gets rid of this horrendous bias. 297 00:34:46,910 --> 00:34:55,040 So because of the crude but we are missing 66 missing we're missing ten in 66 girls in the control group. 298 00:34:55,370 --> 00:34:58,759 And these are predominantly from from socioeconomic group. 299 00:34:58,760 --> 00:35:01,760 So that's a higher rate of pregnancies. 300 00:35:02,270 --> 00:35:08,540 So if the pregnancy rate was 50% or greater among these missing girls, then there's no difference between the two groups. 301 00:35:08,990 --> 00:35:18,140 Now it's 50% pregnancy, very likely. I did try and look for pregnancy rate of girls in Western Australia. 302 00:35:18,530 --> 00:35:23,110 Teenage girls as far as I can find on the internet. There's no routine data published between date. 303 00:35:23,480 --> 00:35:27,530 I don't know what you need to get my social status. 304 00:35:28,580 --> 00:35:38,150 However, I was work on a trial for the AMP trial where we were randomising pregnant teenagers to go to get support from getting pregnant again. 305 00:35:38,480 --> 00:35:47,030 And the pregnancy rate among that group was 66%. So it can be really quite high in some socioeconomic, but not unlikely. 306 00:35:47,030 --> 00:35:53,000 So this is the first calculation of this, so that we can target three one, 307 00:35:53,240 --> 00:36:01,970

3001 and 50 eligible 2100 eligible and simulate a group a difference of 8.45%. 308 00:36:03,140 --> 00:36:14,690 So we're missing 266 girls because we both applied with the total intention to take three population, figure out 158.45% to get 266. 309 00:36:15,710 --> 00:36:17,960 And then we can work out if calculation, 310 00:36:17,960 --> 00:36:30,590 we can then work out the likelihood of what the pregnancy that would have to be to make sure that does not affect all of the simulated interesting. 311 00:36:33,190 --> 00:36:39,350 If you go to that when they did this calculation, which is wrong, nothing's wrong. 312 00:36:42,380 --> 00:36:45,770 We can look at that one figure. 313 00:36:45,770 --> 00:36:54,310 The quote is that the. They expect that to be a 16.8% pregnancy rate. 314 00:36:54,740 --> 00:36:58,000 And this cohort of girls, that's what the historical data says. 315 00:36:58,210 --> 00:37:02,560 On average, 64 ex-employees gave that pregnant. That's what they use in some sort of calculation. 316 00:37:03,070 --> 00:37:06,640 Now the pregnancy rate in the. 317 00:37:08,760 --> 00:37:14,090 Intervention group. And up was 16.6%. 318 00:37:15,350 --> 00:37:19,910 Yeah, 60 left in the control group was 16.6%. 319 00:37:20,160 --> 00:37:24,840 The intervention group was was 15.6%. 320 00:37:24,870 --> 00:37:30,380 So for the infantry group was 16.6%. So the intervention group, which had the highest compliance rate, 321 00:37:30,740 --> 00:37:35,960 had a pregnancy rate that was equal to the historical pregnancy rate average, virtually identical. 322 00:37:36,470 --> 00:37:42,200 The intervention group had a lower pregnancy rate. So why would they why would the intervention so much lower? 323 00:37:43,250 --> 00:37:48,620 It's because of selection bias. So. 324 00:37:52,260 --> 00:37:57,180 To symbolise the claustrophobia. A huge proportion, just a child. 325 00:37:57,810 --> 00:38:01,620 Since I've been working with that, it's terrible. 20 years ago. 326 00:38:01,770 --> 00:38:06,929 That's still terrible. You still can you still get that baby simulator. 327 00:38:06,930 --> 00:38:16,590 So if you try, you try to publish a random individual Monsanto trying to into that reject my letter pointing out the problem with this trial. 328 00:38:17,570 --> 00:38:30,389 I've got no athleticism but in that at least the New England Journal of that so they are but it seems to be I don't know why journals 329 00:38:30,390 --> 00:38:37,170 publish well high profile journals publish I mean should publish this child should publish as a letter of warning to history. 330 00:38:37,290 --> 00:38:43,770 This is yet another fact we designed to stop, but we don't publish it, that we can actually take any meaning from them. 331 00:38:45,030 --> 00:38:48,000 So what should you do about it? 332 00:38:48,150 --> 00:38:55,410 We should try and do individual randomised to go farther for possible and a lot of people do trust the trial to avoid contamination. 333 00:38:55,890 --> 00:38:59,280 So when I was sitting on a one of the study board, 334 00:38:59,640 --> 00:39:11,640 the stated it was the trial which I did my best to stop because they wanted to do cluster randomisation to try and reduce contamination. 335 00:39:11,790 --> 00:39:17,220 So yeah. So after reducing this pattern thinking patients might talk to each other. 336 00:39:17,370 --> 00:39:27,590 Well, I don't know about you when I go to my GP. If I go to my GP because I'm ill, I get the GP but I'm waiting for five of of sick people. 337 00:39:28,010 --> 00:39:31,400 The last thing I do is talk to not because I'm particularly uncertain which I am, 338 00:39:31,730 --> 00:39:38,310 but because I've got the lurking and I don't want to come out with what I've got and what they've got as well as I sit well away from them. 339 00:39:38,320 --> 00:39:42,950 So I talk to the best patients. 340 00:39:43,100 --> 00:39:44,210 They wanted to do just that. 341 00:39:44,370 --> 00:39:53,030 I said, Well, anybody who's had any had a child knows the the social group of young mothers is not wanting from their GP practice. 342 00:39:53,040 --> 00:40:03,779 It tends to be women better than 50 classes. Or that's the group that my wife had as a sort of group that came from lots of different GP practices. 343 00:40:03,780 --> 00:40:09,440 That contamination is an overrated reason for using cluster randomisation. 344 00:40:10,730 --> 00:40:19,550 So here's an example of what they wanted to do, but that is some kind of public health intervention from 1970, 345 00:40:20,240 --> 00:40:27,920 and they wanted the heart to run twice 2000 participants instead of 1282 because of clustering. 346 00:40:29,000 --> 00:40:32,390 So another way of looking at the randomised 2000 people, 347 00:40:32,810 --> 00:40:40,850 one than an individual randomised trial and said what could we have detected so that we could have detected a 7% reduction in smoking. 348 00:40:40,850 --> 00:40:45,500 Right. So let's assume that is a 9% reduction in smoking, 349 00:40:46,610 --> 00:40:52,639 but we actually saw a seventh reduction that because 20% of the sample contaminated and that's assumes 350 00:40:52,640 --> 00:40:58,970 the contamination is as effective as being delivered by the health promotion field the same time. 351 00:40:59,360 --> 00:41:05,980 So that's not likely to happen in particular set of the cup point. 352 00:41:07,590 --> 00:41:16,910 So if you get if you are pretty sure if you get contamination above 30%, then you can trust the randomisation to avoid the contamination effect. 353 00:41:19,730 --> 00:41:23,240 So is it possible when we looked at this 24th draft, 354 00:41:23,240 --> 00:41:31,640 only the third individual randomised trial and the responses to this technique will also comply, which causes best case analysis? 355 00:41:31,870 --> 00:41:34,390 What's the instrumental variable? Technically, 356 00:41:34,400 -->

00:41:49,610 using the RANDOMISATION instrument that can give you a high estimate that if not an unbiased dilution effect of the mass contamination and again, 357 00:41:50,060 --> 00:41:57,310 it has the same powers and I treat it as a as an ordinary analysis if the contamination is central. 358 00:41:58,790 --> 00:42:11,110 So if you still want to go out first and three randomised before randomisation so that you could use it to address the trial on that. 359 00:42:11,330 --> 00:42:18,290 So what you would do is you would approach the schools or 57 schools before you randomise them just as you would do it, an individual randomisation. 360 00:42:18,680 --> 00:42:22,670 You would contact the girls and their parents to get consent to go in trial. 361 00:42:23,480 --> 00:42:37,130 Then once you've got consent, they will 3000 odd thousand and 5000 or so altogether, you might say, okay, 3000 of them in that test. 362 00:42:38,420 --> 00:42:45,350 Once you've got the consent in the bag, you've then randomised the schools and those girls consented in the intervention. 363 00:42:45,350 --> 00:42:55,550 Schools are invited to participate in banged on about the here and the UK being which would have done that. 364 00:42:55,640 --> 00:43:03,230 What we could have done is identify all people with back pain in the fifth vote and ask them if they 365 00:43:03,230 --> 00:43:09,810 would want to take part study where we observe that the severity of their back pain over time and yeah, 366 00:43:09,980 --> 00:43:17,030 chunk of them would have said yes once we've got them in, we would have them at the bypass and then followed them up the shoulder. 367 00:43:17,030 --> 00:43:26,420 Pain trial. We couldn't have done that because it was an incident condition, so you couldn't so you can do it with prevalent conditions. 368 00:43:26,630 --> 00:43:31,430 So the pregnancy stage is a condition that's going to happen. 369 00:43:31,430 --> 00:43:35,660 So you can recruit the girls a long time before the event occurs. 370 00:43:35,870 --> 00:43:41,720 But if you've got five shoulder pain, you need a treating there and then you can't do prevalent recruitment. 371 00:43:42,230 --> 00:43:48,260 So the two approaches could deal with it. First, you can try to match the person doing the recruiting. 372 00:43:48,260 --> 00:44:00,620 So a trial of mental health intervention to try and improve the care of patients with depression and train the GP receptionist to attend, 373 00:44:00,980 --> 00:44:02,090 to speak patients, 374 00:44:02,090 --> 00:44:13,460 to come in and to identify those that may be suffering from depression and then ask them to go to trial before they from that same the same technique. 375 00:44:13,700 --> 00:44:21,010 So the hope was by not by using a receptionist to do that, it would alleviate the. 376 00:44:21,420 --> 00:44:25,840 But then they couldn't do enough to show off and showed us the trial time. 377 00:44:25,920 --> 00:44:34,080 We could we could have done that. What actually cast aspersions on the people during the show came from some ways because what they did, 378 00:44:34,680 --> 00:44:43,080 they originally proposed to MRC this cunning design as a split prop design because basically you randomised 379 00:44:44,550 --> 00:44:55,330 to a group interventions to check whether treatment by changing GP is is better than an infection. 380 00:44:55,450 --> 00:45:01,290 But we can't compare that because these guys, as we can see on that recruitment graph, 381 00:45:01,410 --> 00:45:06,290 these guys are recruiting different sets of patients more to perform a certain equipment set. 382 00:45:06,310 --> 00:45:11,310 So what we do is a patient pitches of cells and at this point they're then randomised to see their 383 00:45:11,320 --> 00:45:16,020 GP or randomise to go and see the mythologies because the rheumatologists are doing the training. 384 00:45:16,260 --> 00:45:26,280 So you would expect if the GP you want to get the GP capability dealing with this problem centrefold and the control group, 385 00:45:26,280 --> 00:45:36,270 you do the same thing and get you involved in talking to right now if the GP care is the same because we've been told it's an intervention group, 386 00:45:36,810 --> 00:45:39,750 these guys that the outcomes are the same. 387 00:45:40,560 --> 00:45:47,219 And also the fact in that group is another effect which is that the GP is as good as the rheumatologist anyway. 388 00:45:47,220 --> 00:45:56,310 Both rheumatologists and trainees not adding anything on to his effort is is equal to still equal. 389 00:45:56,460 --> 00:46:02,730 But but what you'd expect of the intervention was was better in the GP care would be better. 390 00:46:03,030 --> 00:46:08,550 That is a positive result that the GP thought during the training were. 391 00:46:11,880 --> 00:46:16,560 Okay. So when is it not a problem? 392 00:46:17,250 --> 00:46:23,510 So it's not a problem if you if you say the allocation is a lot, if you alter behaviour. 393 00:46:23,520 --> 00:46:32,729 So for example in the U.S. they did under my hospital a year, two different countries pay different, different hours of work. 394 00:46:32,730 --> 00:46:41,250 So junior doctors were on what you do on long hours or the hours were curtailed and the intervention groups that worked on that 40, 395 00:46:41,250 --> 00:46:47,760 50 hours a week and the outcome with mortality in the hospital. 396 00:46:48,070 --> 00:46:51,140 And so in that case, patients would be sent to hospitals. 397 00:46:51,810 --> 00:46:58,200 The patients don't know the hospital visits. It's unlikely patients who get referred know it by whoever is the first hospital, 398 00:46:58,800 --> 00:47:04,060 by the knowledge, even if they knew that the doctors are being paid that way. 399 00:47:04,110 --> 00:47:05,600 But that's in light of the fact. 400 00:47:05,880 --> 00:47:16,050 Similarly, if you run out of randomised compulsive flu vaccinations to try and reduce flu related mortality among their patients, 401 00:47:16,350 --> 00:47:23,559 again, patients are unlikely to know that happening and patients are unlikely to be admitted to not be in this hospital. 402 00:47:23,560 --> 00:47:34,470 On the knowledge that that there's a trial for that and likely selection bias, this is possible and very enlightening so. 403 00:47:38,170 --> 00:47:44,950 So to sum up so that the boss said this is a huge problem, individual and box office continues to be a problem. 404 00:47:45,850 --> 00:47:51,580 I think the problem, worse and worse, the staff and general staff seem to do it. 405 00:47:52,210 --> 00:48:04,620 I think I look back and I remember the last people who got them off in order to get out of the money that comes along in the work and 406 00:48:04,630 --> 00:48:16,690 plus the staff and was run by politicians and all they had to do was grinding on about how to deal with people cluster level techniques, 407 00:48:16,690 --> 00:48:22,070 frailty, analysis, and all the stuff that normal people like me don't understand. 408 00:48:22,170 --> 00:48:29,950 I think it doesn't matter, you know, if not, if you still doing a biased cluster chart, does it matter which technique you used to do the analysis? 409 00:48:30,280 --> 00:48:36,460 It isn't an answer that anywhere near the truth you catch up to question.