Why randomized controlled trials are crucial for evaluating COVID-19 treatments
Article by Alexis Phillips & David Fajgenbaum, MD, MBA, MSc
Infographics by Ania Korsunska
In the first few months of the COVID-19 pandemic, physicians and researchers scrambled to find drugs that might help COVID-19 patients based on their knowledge of drugs’ mechanisms, preliminary results from laboratory studies, and their emerging understanding of the disease. Dozens of treatments were tried within weeks and many of these treatments looked very promising as large proportions of patients improved on them. Some of these treatments were even touted as cures.
As much as we all hoped that these drugs were effective, there was a major flaw that many were overlooking: the vast majority of individuals with COVID-19 recover without any treatment at all. Therefore, a patient’s recovery on the drug may be unrelated to the treatment. Take the following tongue-in-cheek scenario: giving COVID-19 patients a daily dose of Skittles or having them suck their thumbs would result in a greater than 90% survival rate, not because skittles or thumbsucking is an effective treatment, but because the “natural history” of COVID-19 is such that most people will improve without any treatment. In contrast to COVID-19, idiopathic multicentric Castleman disease (iMCD) is a condition where 0% of patients improve without treatment. Thus, when a patient with iMCD gets a treatment and improves, we can feel confident that the treatment is effective. The tongue-in-cheek example of Skittles being used as a treatment for COVID-19 shouldn’t be misinterpreted as us saying that COVID-19 is not important or serious. Even if a disease has a high survival rate, it is still very serious if it affects a large population and/or if there are lasting consequences among survivors. We see both with COVID-19. Therefore, it is essential that we separate out the effective from the ineffective treatments.
The only way to determine if a treatment works for a disease like COVID-19 is to randomly assign patients to either receive the treatment or to receive no treatment in what’s called a randomized controlled trial (RCT). If the patients randomly assigned to receive treatment have better outcomes than those randomly assigned to receive no treatment then the treatment likely works. Unfortunately, many of the treatments that appeared effective initially based on recovery rates have turned out to not be effective when tested in clinical trials. In fact, of the more than 500 drugs given to COVID-19 patients recorded in the CORONA registry, fewer than five are currently recognized by the National Institutes of Health (1) and Infectious Disease Society of America (2) as having sufficient evidence in support of their use.
The importance of RCTs in assessing effectiveness explains why we have shifted so much of our attention in the CORONA project towards extracting and assessing results of RCTs. We are currently working with the NIH and FDA to leverage data in CORONA to identify the most promising COVID-19 treatments to be included in large RCTs. Below, we summarize the different types of study designs and present examples of treatments that were found to have limited efficacy in treating COVID-19 patients despite their initial popularity. These examples demonstrate the importance of performing RCTs when evaluating clinical interventions.
There are a few key questions that can help to guide one’s assessment of whether a study’s conclusions can be applied to large populations of patients:
- Is the study observational or interventional? In an observational study, the researchers observe the relationship between a variable and an outcome without performing an intervention or deciding which study participants are exposed to that variable. Examples of observational studies include case reports, cohort studies, and case-control studies. Interventional studies involve giving a treatment directly to a patient and assessing how the patient responds.
- Is the study a small pilot or a large trial? The larger the sample size, the more likely the results are real and generalizable. The results of smaller studies are less reliable because they have a higher likelihood of being due to chance.
- Is the study double-blind, single-blind, or open-label? In a double-blind trial, neither the participants nor the investigators know whether a participant has been assigned to the intervention group or the control group. In a single-blind trial, the participant does not know what group he or she has been assigned to, but the investigator has this knowledge. In an open label trial, both the researcher and participant know what group the patient has been assigned to. This is important because sometimes patients will report fewer symptoms if they know they are receiving a treatment or their physician will be less likely to escalate care because they expect the intervention will work.
- Is the study randomized or nonrandomized? In a randomized trial, participants are randomly assigned to study groups. In nonrandomized trials, participants may be assigned to groups based on certain baseline characteristics that may affect the outcome of the intervention. Randomization helps to evenly distribute these characteristics between groups.
- Does the study have a control group? What kind of control group is there? A control group serves as a standard for comparison to the intervention group. Ideally, there should be no significant differences between the control group and intervention group. Including a control group allows researchers to ascertain that an observed effect in the experimental population is actually due to the intervention and not a part of the natural history of the disease under investigation. In some trials, the control group receives a placebo that is indistinguishable from the drug that is under investigation but that causes neither harm nor benefit. In other trials, the control group does not receive a placebo, but receives a standard of care without the study drug.
A large, multicenter, double-blind placebo-controlled randomized controlled trial (RCT) is the gold standard of clinical trials. RCT results are more persuasive and generalizable than the results of trials that are non-randomized or observational because they help to guard against several kinds of bias and confounders.
While observational and in vitro studies don’t inform us of whether a treatment is effective or not, they can be helpful in prioritizing promising drugs for more rigorous testing in RCTs. Dexamethasone is a good example of a treatment that first showed a positive signal in observational data (which we noted in CORONA) (3) and was subsequently found to be effective in a large RCT (4). Tocilizumab also looked positive in observational data and had an unclear benefit in RCTs until multiple large RCTs of severely ill COVID-19 patients demonstrated its effectiveness in those specific patients. However, there were many other drugs that showed promise in laboratory studies, case reports, and observational studies, such as hydroxychloroquine, lopinavir/ritonavir, and azithromycin, that have since had several RCTs performed that failed to show benefit. These medications serve as examples of why RCTs should be prioritized in terms of both resource allocation and clinical decision making.
Hydroxychloroquine
Hydroxychloroquine is an antimalarial that rose in popularity early in the pandemic thanks to promising results in early in vitro studies (5, 6) case series, and observational studies (7), as well as endorsements from public figures. In March of 2020, the US FDA issued an emergency use authorization for the use of hydroxychloroquine in COVID-19 patients that was later revoked in June after emerging data from several RCTs determined it is highly unlikely to be effective and may actually be associated with adverse effects. As of March 2021, 18 RCTs of hydroxychloroquine have been completed and only 1 shows benefit. Several other RCTs have failed to show benefit of hydroxychloroquine to prevent infection after exposure (8), treat COVID-19 outpatients (9), decrease viral load (10), or to treat patients in the mildly (11), moderately, or severely ill (12) subgroups.
Azithromycin
Azithromycin is a macrolide antibiotic that was hypothesized to have therapeutic effects in COVID-19 patients based on observational and in vitro studies linking the medication to positive immunomodulatory effects and decreased SARS-CoV-2 viral load (13, 14). It has since shown disappointing results in RCTs evaluating it as a standalone agent and in combination with other agents, most commonly hydroxychloroquine (HCQ). RCTs of COVID-19 patients with severe (15) and mild-to-moderate (16) COVID-19 failed to demonstrate significant improvement in clinical status relative to the placebo group. So far, only one of four published RCTs support the routine use of azithromycin as an antiviral agent in patients with COVID-19, although its utility in treating secondary bacterial infections is still unclear.
Lopinavir/ritonavir
Lopinavir is a protease inhibitor that is combined with ritonavir to increase plasma half life. Lopinavir/ritonavir was suggested as a treatment for COVID-19 due to studies showing in vitro activity against SARS-CoV-2 (17). However, it has not shown efficacy in any of the 6 RCTs of hospitalized COVID-19 patients completed thus far.
With vaccines not yet widely available, the COVID-19 pandemic remains a looming threat that requires evidence-based therapies. There is an urgent need to more fully explore the efficacy of potential therapies for COVID-19 in well-powered RCTs.
Data accurate as of date of publication (3/25/2021). Featured image Illustration by Julia from Icons8. Other icons in infographics also from icons8.com.
Disclaimer:
CORONA Insights and the CORONA Treatment Registry are for informational and exploratory purposes only and are not intended to be used as medical advice. This site is intended to facilitate the exploration of therapies mentioned in COVID-19 medical literature. Users assume full responsibility for use of the information on this site and understand and agree that the CDCN and its third party content providers are not responsible or liable for any claim, loss, or damage (including personal injury or wrongful death) resulting from its use. Reliance on any information provided by the CDCN or third party content providers is solely at your own risk. Our extractors do not assess the scientific merits of these results and conclusions. We disclaim any warranty concerning the accuracy, timelessness, and completeness of information on this site. This site also contains links to external sites. The CDCN is not responsible for the content and does not make any representations regarding their content, timeliness, or accuracy.