6 • To understand the distinction between experimental and nonexperimental designs. • To understand what is meant by bias and confounding. • To understand the key components of a randomized controlled trial. • To understand the rationale for randomization. • To understand the rationale for blinding in randomized controlled trials. • To learn how to critically appraise the results of a randomized controlled trial. The randomized controlled trial is a true experiment as the investigator manipulates the intervention received by the subject in a controlled manner. The intervention received is also determined by a random process and not by other factors. These attributes allow for optimal assessment of the causal relationship between the intervention and the outcome. This concept is known as internal validity. Consider an alternative situation whereby the investigator defines a number of treatment paths for a particular disorder. However, the choice of treatment is up to the clinician caring for the patient. At the conclusion of the study, the investigator compares the results of the various treatments. This is a quasi-experimental design with lower internal validity than the randomized trial. The reason is that if there turns out to be a difference between treatments, it may be related to the factors that influenced the choice of treatment received rather than the treatment itself. This concept, known as confounding by indication, has already been discussed in Chapter 5 as well as bias, and is discussed in more detail later in this chapter. While quasi-experimental designs may be used to assess the causal relationship between treatment and outcome, the level of inference (i.e., internal validity) is lower than for a randomized design. While the goal of research is to learn the “truth,” bias refers to any factor that causes the results of a study to deviate from the “truth.”1 Bias does not imply malice, rather it is generally the result of a systematic error in research methodology. It is often difficult to predict the impact that bias may have on the results of a study. Bias may not be able to be addressed statistically. It is critical to understand the potential sources of bias when designing a randomized controlled trial in order to avoid them. We consider some of the more common forms of bias that affect interventional studies and later highlight aspects of the randomized design that attempt to address these issues. Selection bias, also known as sampling bias, occurs when enrolling subjects into the trial.1 Selection bias occurs when only a limited subset of potentially eligible subjects is enrolled in the trial. While it is never possible to enroll all potentially eligible subjects in a trial, selection bias occurs when there is an important difference between those who are enrolled and the overall group of those who are eligible to be enrolled. For example, if patients with severe arthropathy decline participation in a trial more often than those with mild arthropathy due to physical limitations in coming to the hospital for imaging tests, the trial population will have a disproportionate larger number of patients with mild disease. This will alter the spectrum of imaging findings toward mild arthropathy. This difference in participants may have important implications when assessing the results of the study as the trial population does not reflect the target population to whom the results of the trial are intended to be applied. Ascertainment bias, also known as workup or verification bias, refers to the situation whereby the results of a trial are distorted by knowledge of which intervention the subject is receiving.1 This may occur when administering the trial intervention or by co-intervention, which refers to administration of other treatments in addition to the trial intervention. For example, in a trial of endoscopic versus image-guided gastrostomy tube placement, clinicians taking care of the procedure may assume that patients undergoing the image-guided option will have less pain and therefore may be less likely to prescribe narcotic analgesia. This may result in more pain in the patients who underwent the image-guided procedure for reasons (i.e., underdosing) other than the procedure itself. Ascertainment bias may also occur during assessment of outcomes. For example, in a trial of ultrasound vs. computed tomography for diagnosis of appendicitis in children, for ethical reasons the surgeons will review the results of the imaging studies prior to deciding on the patients’ management (surgery vs. observation) which may influence their management decisions. Blinding is a procedure that may be undertaken, whenever ethically possible and feasible, in order to limit ascertainment bias. Trials with positive results are more likely to be published.2,3 Therefore, literature reviews and meta-analyses may result in biased (typically overly optimistic) estimates of a treatment effect or diagnostic accuracy of imaging tests. In order to limit this bias, most journals require, prior to consideration for publication, that the trial be registered at its outset on a public trial register such as www.clinicaltrials.gov. Therefore, even if negative trials are never published, authors of reviews will be aware of their existence and can contact the investigators for results. Moreover, in the conduct of a review, restriction of language of manuscripts to English simplifies its performance but reduces the generalizability of the review results to non-English-speaking countries.4 A confounding variable is a factor that distorts the true relationship of the study variable of interest because it is also related to the outcome of interest.5 Confounding occurs when the impact of an intervention is altered because of the association of the intervention with other factors that influence the outcome. The confounding variable may mask an actual association or it may falsely demonstrate an apparent association between the study variables where no real association between them exists. One of the most prevalent types of confounding is confounding by indication. Confounding by indication refers to the situation whereby patients with a particular prognosis are allocated preferentially to a particular treatment or intervention arm. These patients are systematically different from those who receive alternate treatments. This systematic difference leads to apparent differences in outcome between the treatments or interventions. However, the observed differences are due to the confounders rather than the therapies or interventions received. Fig. 6.1 illustrates confounding by indication in a hypothetical nonrandomized study that examined survival of patients undergoing endovascular compared to open aortic aneurism repair. The results demonstrated that 5-year post-procedure survival in the endovascular group was worse than with open repair. However, when you look closer at the data you discover that older and sicker patients underwent the endovascular procedure, whereas younger patients with less comorbidity underwent open repair. Since both age and comorbidity influence overall survival (the outcome), independently of the procedure performed, they are confounders. In this example, it is possible that the difference in the outcomes between the treatments is related to these confounders rather than inherent differences in the procedures. It is difficult to predict the impact of various confounders, which may exaggerate or diminish treatment effects. As well, it is impossible to identify all confounders. Therefore, even if an investigator is able to identify and account for confounders in the research design, there will always be unknown confounders that may influence the results. The main strength of the randomized trial is that the random assignment to treatment groups leads to a balanced distribution of both known and unknown confounders across treatment groups.6 As a result, the randomized controlled trial is able to assess the impact of a particular intervention while minimizing the influence of confounders. Fig. 6.2 presents a basic outline of a randomized trial. The trial depicted in this diagram is the simplest type of randomized design, the parallel group design.7 In the parallel group randomized controlled trial, there is a single experimental intervention that is compared to a single control intervention. Busse et al. provided an overview of more complex trial designs.7 The key components of the trial—study sample, randomization, intervention, and outcome assessment—are addressed in the following sections. The study sample refers to the subjects who participate in the trial. A key feature of a randomized controlled trial is that all subjects must be able to receive any of the interventions that are being studied as part of the trial. In order to do this, researchers develop eligibility (inclusion and exclusion) criteria. When developing eligibility criteria, there are a number of factors that the investigators need to balance.8 First, the investigators need to decide who the target population for their trial will be. This is typically the patient population to whom they would like the results of the trial to apply. Second, investigators need to consider how feasible it will be to recruit subjects who meet their eligibility criteria. If the trial is being performed in a rare disease, then the investigator will need to consider enrolling subjects at multiple sites in order to meet their recruitment target within a reasonable period of time. Usually the broader the eligibility criteria, the easier it will be to recruit. However, the more dissimilar subjects are from one other, the more heterogeneous the treatment response will be. Increasing heterogeneity results in a need for a larger number of subjects in order to answer the research question. Finally, investigators need to consider subject safety in developing their eligibility criteria. The choice of eligibility criteria has important implications when applying the results of the trial. The more similar the trial subjects are to a prospective population of patients, the more likely that the results will be applicable to those patients.8 This concept is known as generalizability or external validity. When applying the results of a randomized trial, it is important to not only look at the eligibility criteria as described in the methods section of the paper, but also the characteristics of the subjects who actually participated in the trial. If there are important differences in the characteristics of participants compared with the target population, then the trial may have been subject to a selection bias that may alter the conclusions of the trial. Randomization is the process of assigning subjects to groups based on chance.9 Generation of the randomization sequence refers to developing the order to which subjects will be assigned to treatment groups. There are a number of methods for randomization, but regardless of the method used, it is essential that subjects are assigned to treatment groups based on a random process and not another, potentially predictable, factor. For example, some trials have used hospital file numbers, birth date, or day of the week to assign subjects to intervention groups. While the relationship between these factors and the patient is random, since these factors are known prior to enrollment they may influence the decision to enrol a patient into the trial. If this were to occur, the trial is subject to allocation bias, which is a subset of selection bias. Appropriate methods for randomization range from low-tech approaches such as dice or random number tables to sophisticated computer-based algorithms.10 Generation of the sequence to which patients will be assigned to the trial is often done prior to the start of the trial but may be done on an ongoing basis when computer-based approaches are used. When completely random methods, such as a random number table, are used to generate the randomization sequence, the treatment group sizes may be quite unequal in smaller trials just due to chance. Block randomization is a technique of randomization used in smaller studies to ensure more equal distribution of subjects across treatments. Blocking is achieved by creating blocks of a size that are multiples of the total number of treatments (e.g., AABB, BAAB, or BACACB, BACBCACAB). In each block, there is an equal number of subjects in each group but the order of treatments in the block is determined randomly. Therefore, after each block is complete, there will be equal numbers of patients in the treatment groups. While blocking seems to be a good solution of the problem of unequal group sizes, the problem with blocking is that since the randomization sequence is broken into small components that all have an equal number of subjects from each group; it may become possible to predict the next subjects toward the end of a single block. Strategies to avoid this include varying block sizes and keeping the block size concealed from the investigator. The goal of randomization is to ensure that confounders are randomly distributed across the treatment groups. This does not necessarily mean that the groups will have an equal number of subjects with specific characteristics although as trials become larger, then the likelihood of this increases. Sometimes, investigators will want to ensure that they have sufficient numbers of subjects with particular characteristics across the treatment groups. This may be done if the investigators are planning on performing subgroup analyses using these characteristics or if they want to ensure that each treatment group represents the spectrum of disease. Stratification is a process whereby subjects are randomized and allocated to groups based on key characteristics to ensure that there are sufficiently large subgroups of patients with particular characteristics so that particular issues can be addressed with sufficient power. The simplest approach to stratification would be to develop different group assignments for each combination of subject characteristics—known as strata. For example, if the investigators wished to stratify by gender and disease severity, they would require four randomization tables (male/mild, male/severe, female/mild, female/severe). The major issue with stratification is that the number of sequences needed increases exponentially with the number of stratification variables.11 Stratification leads to methodological complexity as each combination of strata forms essentially a separate randomized trial that needs to be accounted for in the analysis. But as the number of strata increases, the number of subjects within each stratum decreases for a fixed trial size and it is possible that some strata will have only treated or untreated patients. This also has implications for subgroup analyses as the study will become underpowered for these analyses because of the low number of subjects within the various strata.11 For this reason, one should generally not stratify on more than two variables, and in larger trials, stratification may not be necessary. Regardless of the method used to generate the randomization sequence, it is essential that a potential subject or investigator not be able to know or predict the group assignment prior to enrollment in the trial.12 Such knowledge may influence who participates in the study which may introduce allocation bias whereby different patients may be enrolled or not enrolled depending on the knowledge to which group they would be assigned. While central randomization may seem to be ideal, it can be costly especially for trials that enroll subjects at unusual hours of the day. Envelopes are used by many investigators as they are a simple way to reveal the group assignment following enrollment. However, use of envelopes is controversial as an approach to allocation concealment. If envelopes are used, it is essential that they are opaque, sealed, and sequentially numbered and cannot be resealed once opened. This ensures that there is no way to determine group assignment until the envelope is opened. Also, once the envelope is opened the subject is considered to be enrolled and assigned to a particular group and another envelope cannot be chosen. The importance of allocation concealment was highlighted by Shultz et al., who reviewed the results from a number of meta-analyses looking at impact of methodological issues on estimates of treatment response. In trials where allocation concealment was not performed, the trial overestimated the treatment effect by 40%. In trials where it was unclear how allocation concealment was achieved, treatment estimates were 30% higher.13 Therefore, it is essential that randomized trials have adequate allocation concealment procedures and that these procedures are described in the publications reporting the methods and trial results. While allocation concealment protects from knowledge of group assignment prior to enrollment in the trial, blinding keeps the group assignment secret after enrollment has occurred.14 Blinding usually lasts for the duration of the trial, although some trials maintain the blinded group status until after the trial has been analyzed. Blinding limits ascertainment bias, which occurs when knowledge of group assignment influences the delivery of the intervention or assessment of outcomes. Controlling for other factors, when compared with blinded trials, unblinded trials overestimated treatment response by 17%.13 While it may seem optimal to blind all randomized trials, it is not always feasible. For example, it may not be possible to blind patients undergoing a trial of an image-guided versus a surgical procedure. However, it may still be possible to blind outcome assessors. Preserving blinding may also be ethically challenging, such as the notion of using a sham procedure in a trial comparing an invasive procedure to a noninvasive procedure. The decision to blind should be based on the type and magnitude of bias that would be introduced if the trial were not blinded as well as the feasibility of blinding. This depends on the specific intervention that is being studied as part of the trial and also the outcome measures used. In general, subjective outcomes are more prone than objective outcomes to bias from lack of blinding. Blinding can be done for different groups involved in the trial, namely, participants, care givers, outcome assessors, and statisticians. Frequently one sees the terms single, double, or multiple blind. Single blind refers to one of the groups being blinded; double, two groups—usually subjects and investigators. Multiple blind refers to multiple groups. Since these terms are not useful for a reader in knowing exactly who was blinded as part of the study, it is preferable to describe exactly which groups were blinded and how rather than using these terms. The intervention refers to the treatment, procedure, or drug being evaluated in the trial. The intervention is compared with the “control” treatment which may be another medication, modality, procedure, nonsurgical approach, or placebo.15 When designing a trial the intervention needs to be clearly defined and described so that it is clear exactly what was done, partly so that the trial can be replicated by others. The choice of interventions to be studied in a randomized controlled trial is typically based on equipoise.16 Equipoise classically is defined as expert opinion that the treatments being considered in the trial have equivalent efficacy. Given that the classic definition of equipoise is quite stringent, community equipoise or uncertainty has been suggested as the basis for a trial. By this alternate definition, trials are justified when there is a difference of expert opinion with regard to the relative merits of the treatments or experts are uncertain which treatments are best. Since health care cannot be compromised by participation in a trial, all treatments must have sound rationale for their use and treatments with good evidence of benefit or harm relative to another cannot be compared to one another in a randomized trial. Outcomes refer to the measures taken at the end of the trial that will be used to assess whether the intervention and control groups differ (i.e., that one treatment is better than the other). The choice of outcome depends on the objectives of the study.17 There are a number of types of outcome measures including clinical outcomes (e.g., recurrence of stroke, length of stay in hospital), biologic outcomes (e.g., hemoglobin), patient-derived outcomes (e.g., quality of life or satisfaction), and economic outcomes (e.g., cost). While trials will typically have a number of outcomes, one is designated as the primary outcome. This should be the outcome that addresses the primary question that the trial aims to answer. This is also the outcome on which sample size calculations are based. Whatever the outcome measure used, it is essential to adequately define it. The importance of this can be demonstrated from a study of a number of definitions of surgical site infection.18 Depending on the definition used, the overall infection rate varied from 7% to 20%. When choosing an outcome measure and its definition, it is important to consider the implications that this will have when applying the results of the trial. For example, the observed complication rates will be most accurate in predicting risk of complications outside the trial, if they are defined the same way as the trial defined them. Clinical trials can be challenging studies to perform given their cost and complexity. It is mandatory that investigators collaborate with a statistical and methodological expert throughout the trial to ensure that the trial is able to achieve its stated objectives. This is particularly important during the design and analysis phase of the trial. Subjects in clinical trials may drop out from the trial, or receive the treatment that is being used in the other group (known as “cross-over”). Dropouts and cross-overs pose a challenge when analyzing the results of a randomized trial as reasons for crossing over or dropping out are generally not random.19 As a result of this, treatment groups that were comparable at the start of the trial (because of randomization) will be systematically different from one another at the end of the trial due to dropouts and cross-overs. Removing noncompliers may destroy the unbiased comparison provided by randomization. Unless appropriate analytic approaches are used, bias will be introduced. The intention-to-treat analysis is a conservative strategy for analysis of randomized trials and, therefore, the preferred method for dealing with dropouts and patients who cross over.19 Subjects are analyzed according to the group to which they were originally randomized regardless of the treatment they have received. Using this strategy, investigators include all patients at the study entry, regardless of whether they received the treatment or intervention to which they were randomly allocated, subsequently withdrew from the study, or deviated from the study protocol.20 Two alternatives to intention-to-treat are the “as-treated” analysis, where patients are classified according to the treatment actually received, and the per protocol analysis, where patients are included only if their treatment went according to study protocol (for dose, timing, compliance, etc.). The intention-to-treat principle maintains the benefits of randomization minimizing the influence of withdrawals, noncompliers, and patients lost to follow-up and provides an estimate of treatment effectiveness that more closely reflects the “real-world” effect of prescribing a treatment. An advantage of this strategy is that it allows greater generalizability of study results and minimizes the risk of a type I error (“to say that there is a difference between groups when in reality there is not”) because of its greater cautiousness. A limitation of this strategy is that such an analysis is less likely to show a positive treatment or intervention effect, especially in studies that randomize patients who have little chance of benefiting from the intervention.21 For example, in a drug trial, where patients may switch drugs due to side effects, the intention-to-treat analysis will provide an estimate of the overall effectiveness of the drug that takes into account the fact that not all patients will be able to take the drug.22 Fig. 6.3 illustrates a hypothetical example of how excluding patients who do not receive the intervention to which they were originally assigned can introduce bias. Imagine a randomized trial of 200 patients who require the use of peripherally inserted central catheters (PICCs) for different clinical indications; 100 are assigned to receive cuffed PICCs and the other 100 to receive uncuffed PICCs. As per the protocol, the PICC is to stay in for a minimum period of 1 week and the primary outcome is the complication rate for PICCs at the end of this period. Let’s assume that the investigators expect uncuffed PICCs to have an equal or a higher rate of complication (infection, malposition, thrombus formation) as compared to cuffed PICCs. In the cuffed PICC arm, 20 out of 100 patients developed a complication, following the 1-week period. In the uncuffed PICC arm, 10 patients withdrew prior to the first week, and of the 90 patients remaining on the trial, 10 developed a complication following the 1-week period. In this example if we restrict the analysis in the uncuffed PICC arm to the patients who had the PICC for the minimum period of 1 week (a per protocol analysis), the event (complication) rate will be 11% (10/90); however, the rate in the cuffed arm will be 20% (20/100). Since the investigators had assumed that uncuffed PICCs would present with an equal or higher rate of complications, these values represent a spurious (unexpected) result. Alternatively, if we include all randomized patients, according to the intention-to-treat principle, and take a worst-case approach of assuming a complication in those who ended the trial early, we see that there were 20 events in each arm and no impact of cuffing on complications.
Randomized Controlled Trials
Learning Objectives
Concepts
Experimental Design
Bias and Confounding
Bias
Selection Bias
Ascertainment Bias
Publication Bias
Confounding
Overview of a Randomized Controlled Trial
Study Sample
Randomization
Generation of the Randomization Sequence
Block Randomization
Stratification
Allocation Concealment
Allocation Concealment versus Blinding
Intervention
Equipoise
Outcomes
Analyzing and Reporting Clinical Trials
Intention-to-Treat Analysis
Radiology Key
Fastest Radiology Insight Engine
