Welcome back. We have now entered the section of the course where we're going to talk about discrete types of medical research studies, and we're starting with the cream of the crop, the gold standard, the randomized, controlled trial. This is the most expensive and most highly impactful type of study design that we have in the medical sciences. But it might not be all it's cracked up to be. So our goals today are to understand what makes a randomized trial so special. What's the special sauce there? Why is it so important that we do these types of studies? We're going to learn some of the rules for interpreting randomized trials, and we're going to discuss how they can actually go wrong and how you can recognize when they're going wrong. But to give you a good sense of how a randomized trial works, we're going to conduct a randomized trial right now on a very important topic, which is whether I look smarter when I wear my glasses or not. We're here on the beautiful Yale campus on this fine spring day to find out, on a scale of 1-10, how intelligent do you think I am? Probably say an eight and half if you're professor here. How intelligent do you think I am? Nine. On a scale of 1-10, how intelligent do you think I am? Six. Ten. I'm going to say- I'm going to go with five. Seven. Ten on 10. Thank you very much. How intelligent do you think I am? You are? Like a six. Five. Not to offend you, but I would say- Six. If I'm assuming five is standard, I'll give you a seven or eight because you're professor at Yale, so I'm like,"Okay, he must be somewhat bright." Seven. But I don't want to assume 10 or nine because I think that's a bit high. I totally agree. So that was fun and highly informative. I am electing not to wear the glasses at this point in particular, because then I won't be able to see anyone who is filming this lecture right now. But let's talk about a real study now. This is not a randomized trial. I'm going to talk about an observational study now to give you a sense of what's wrong with these types of studies, and how randomized trials fixes the problem. So here's a study that appeared in the journal JAMA Pediatrics that suggested that women who eat more fish during pregnancy were more likely to have obese children when they gave birth. What you can see here, in this little graph, are three lines. Those are three groups of women by how much fish they ate. The women in orange are the women who ate the most fish, and you can look at the child's age, and you can see out at 3, 4, and 5 years, for example, the women who ate a lot of fish during pregnancy had kids with higher body mass indexes. So you can read this study and you could conclude, "Oh, my gosh. You shouldn't eat fish during pregnancy. You're going to have overweight children," But there's probably something tickling in the back of your head right now that suggests this might not be true. This might not be causal. So what's going on there? Well, the study wants you to think that eating fish leads to overweight kids, but that's not necessarily true. Maybe overweight parents have overweight kids, and maybe overweight parents eat more fish and it has nothing to do with the fish. It's just this third factor, the weight of the parents that are linking both of these observations. Now, these third factors are called confounders, and we'll have a whole talk about confounders. But I think it's pretty intuitive in studies like these, observational studies, to think that there might be these factors that you're not taking account of that are linking the exposure and outcome, inducing a relationship where none really exists. Now, how would we fix this? Well, if we wanted to make sure that that wasn't a problem, we would do a randomized trial. What a randomized trial does is, it randomizes the exposure of interest. So in this case, the exposure of interest was eating fish during pregnancy. In the study I just showed you, they just asked women, "How much fish did you eat during pregnancy?" That is not randomized. But what if we took pregnant women and we forced them to eat a lot of fish or not to eat any fish? I don't know if this is ethical, by the way, but let's just say we could do it. So we force them to eat a lot of fish during pregnancy or not eat a lot of fish, and we force them at random. We just flipped a coin to decide who gets to eat a lot of fish and who doesn't, and then we measure the weight in the kids. That is a powerful study design because those third factors, like being overweight, would be evenly distributed between the fish eaters and the non fish eaters, because we assign them completely at random. So the secret sauce of a randomized trial is this. By randomizing, you balance measured confounders between the two groups, the exposed and the unexposed groups, but even more importantly, you balance unmeasured confounders. So you can measure the weight of the parents, for example, and you can even statistically adjust for that. But what about unmeasured factors? Things that you didn't even take into account like whether the parents watched Mr. Rogers as a kid or something like that, or you just didn't think to measure that. Well, if you randomize, it doesn't matter. The percentages of parents who watched Mr. Rogers as a kid will be the same in both groups because you just flipped a coin, it's totally random. Unmeasured confounders, things like Mojo, just these ethereal femoral things that you can't actually measure are balanced in a randomized trial, and that is a very special thing. Now, when you hear about a randomized trial, you often hear a lot of terms attached to it, stuck onto it. For instance, double-blind, randomized, placebo-controlled, clinical trial. So what does that mean? What are we talking? Let's just walk through it. So double-blind. Double-blind means neither their participants nor the investigators know who is getting what. So you have an active drug and you have a placebo, and the placebo looks just the same, and the patients don't know what they're getting, and importantly me, the investigator, I don't know what they're getting either. Because even if the patients don't know, if I know that the patient is getting the active drug, well, maybe I treat the patient a little better. I treat them special, I call them once or twice more often to see how they're feeling, so I can break the randomization in that way. That's what double-blind means. Randomized means randomized. Who gets what is totally determined by random chance. This is the whole point of the randomized trial. It doesn't work if it's not randomized. Randomization, flip a coin, use a computer, whatever you do, it has to be random. Placebo-controlled. So this is important. Not every randomized trial is placebo-controlled. A placebo is a thing that looks like the active agent. So if the active agent is a pill that is green and oval, then the placebo will also be a pill that is green and oval. You've got to be really good about making placebos work right. So a really good trial, they have their placebo, maybe they'll add something into the placebo pill so it tastes a bit medicine-y. It doesn't just tastes like a sugar pill. So it looks like the real pill, it tastes like the real pill. So you really want to balance that to make sure that people aren't able to figure out that they're in the active group or the control group. Not every study has a placebo control, some have usual care as the control. So half the group randomly gets this new drug, and half the group just gets the usual stuff. They might know that maybe you're not blinded in that case. That's fine, and we can still make some conclusions and trials like that, but we have to be a little more careful because maybe people are feeling better because they know they're getting some fancy new drug. This matters more when the outcomes are subjective, like how do you feel? Or are you having headaches? Or how's your low back pain? Not as important if the outcome is death. Okay. Clinical trial is a broad catchall term for we are testing something, we're doing something, we are perturbing the natural order in some way. So you can actually have a clinical trial that is not randomized. You can just say we're going to treat everyone with this new drug, that could be counted as a clinical trial. It's an intervention that changes the natural way things play out. So the key to understanding clinical trials is an acronym called PICO, P-I-C-O. I'll walk through it with you in the context of this trial which looked at intra-articular triamcinolone versus saline on knee cartilage volume and pain in patients with knee osteoarthritis. What the heck does that mean? Okay. So knee osteoarthritis, that's knee pain, the old fashion, getting old your knees start cracking and hurting. Intra-articular triamcinolone is a steroid injection into that knee. A lot of people think that that helps, you inject some steroids, get the inflammation down, the knee feels better. But let's test that. So what they did is they used the steroid injection compared to just a saline injection. In fact, in this study, I believe they wrapped the syringe in an opaque material of some kind, so that you couldn't actually tell if it was the kind of cloudy steroids or the clear saline, as it was going in. So the providers didn't know what they were injecting the patients. So let's go through PICO in this situation. P stands for population, who is being studied? I is the intervention, what is being studied? C is the control, how is the non-intervention group being treated? What are they getting? O is the outcome. What is the important thing you are trying to measure? A good title of a clinical trial is going to tell you almost all of these things. Population, patients with knee osteoarthritis, intervention, intra-articular triamcinolone, control, saline, outcome, cartilage volume and pain. So all of the PICO elements are right there in the title, but we can dig down a little bit more. If you want to look at population, there are two places you're going to look in a clinical trial. The first place is table 1. Traditionally, table 1 describes the population you're studying, and I've reproduced it here. You can take a look if you want to pause, and look at the data. But basically, you can see for instance, the age in the two groups here was 59 in one group, 57 in another group, that's fairly balanced. This is randomized. So these should all be relatively the same, 52 percent women versus 54 percent women. So it just gives you a sense of who was in this trial? They were, as you might expect, slightly older people who have knee osteoarthritis, not that old, just a little bit old, 67 percent white and so on. But there's another place that you need to look at for the population. I think people often overlook this, and this is often figure 1, which is the flow diagram for the study. What figure 1 should show, if the study is done well, is how they got to their final population of patients. So table 1 shows you the characteristics of the final population of patients. But they didn't just pull those people off the street, in fact, they probably brought more patients than that in, and screened them, and tested them to see if they'd be good for this trial. Figure 1 tells you that. So for instance, this figure shows us that 445 patients were assessed for eligibility to be included in this trial, and 305 were excluded. It tells you why they are excluded. Ten had some medical condition, 27 couldn't get an MRI, various other causes. So when you think of the population being studied it, you might think, "Oh, it's people with knee pain, people with knee osteoarthritis." Be careful because out of 445 people they screened for this study, only 140 got into the study. So that means that the results of the study are not necessarily applicable to every single person. Take a close look at figure 1, population. Intervention, pretty easy, you can look in the methods to figure out exactly what they gave. It turned out to be one milliliter of triamcinolone at a dose of 40 milligrams per milliliter. So they'll give you all those details, that sounds fine. The control is one milliliter of sodium chloride, and the outcome was the change in pain score. If you're very curious, what you can see here is that the mean difference in pain score was 0.64 points. You can see that 95 percent confidence interval here ranges from negative 1.6-0.29, which shows that that range excludes the clinically meaningful improvement in pain, which would be plus one point. That's how much you'd want to see to say, "Yes, steroids are good, let's give people steroids." So in fact, we would classify this as a negative study, not an underpowered study, this is a negative study. This study essentially confirms that for this population of apparent patients, steroids don't work for knee osteoarthritis, you're just as well off getting a shot of saline, of saltwater into your joint. So what are the pitfalls of randomized trials? I told you how great they are. Randomization is so good. But be careful, blinding isn't always blind. What does that mean? Well, I might say that it's double-blind because I didn't tell the investigator who was getting what treatment. But if I didn't make my placebo very good, maybe the investigator can figure it out. I can say in an acupuncture study, that I didn't tell the investigator whether the patient got acupuncture or massage when they went into the back room, but the provider might be able to figure it out if they have little band-aids all over their body. So blinding isn't always blind. So figure out if you can tell whether the blinding was maintained throughout the study. Let's reflect back on the number needed to treat. You might remember that when we discussed the Lipitor study in a prior lecture. When you look at the results of a randomized trial, you want to say, not only is this positive or negative, but how many people do I need to treat to benefit one person? That helps you scale whether this intervention is going to be useful in the general population. Finally, loss to follow-up. So be careful, that flow diagram in figure 1 is not only going to show you who got into the study, it's going to show you who they lost track of along the way. In our knee osteoarthritis study, they actually followed everyone, so they didn't lose a single person. But some studies we'll lose 10, 20 percent of the people that were enrolled, they're just gone. They don't know what happened to them. It seems crazy, but in this day and age, people move, people change their phone numbers, they get lost to follow up. If that number is significantly high, you maybe have to wonder about the conclusions of the study. So here, your take-home points, randomized trials give us some of the highest quality evidence we have, and they are easy to interpret. But be careful, there are some pitfalls. Randomized trials aren't perfect, and it's always no single study is going to be the definitive answer on any medical therapy. I'll see you next time.
