8 • To understand and be able to describe the limitations to making optimal clinical decisions. • To know how to interpret a simple decision tree. • To understand the meaning of expected value and be able to calculate this from a simple decision tree. Clinical practice is characterized by many difficult decisions. Health care providers make decisions about the most likely diagnosis for a patient and choose optimal therapies or management strategies. Patients make decisions by choosing among options for diagnostic tests and treatments based on their understanding of potential risks and benefits. Decision analysis is an approach to analyzing decisions that is systematic, quantitative, and explicit. It quantifies the expected benefits of each course of action by combining two types of measures: estimates of the probabilities of events and numerical quantities signifying the value of these events. Clinical decision analysis is a tool that can identify the strategy for diagnosis and treatment of disease that is most likely to achieve desired health outcomes. Decision analysis plays an important role in a variety of disciplines, including engineering, management, and investment decision making. Clinical decision making is complex because critical, complex factors involving risk and prognosis must be weighed in order to choose the best option. Central to determination of risk and prognosis is the concept of probability. The probability of an event measures how likely it is to occur, expressed as a number between 0 and 1 equal to the proportion of possible outcomes in which the event of interest occurs. A probability can be subjective, quantifying an individual’s belief in the likelihood of an event, or based on actual frequency. In clinical practice, the probability of occurrence of an event for one patient is usually based on the frequency of the event in a group of similar patients. The challenges of thinking in probabilistic ways include choosing the correct group of patients with similar characteristics on which to make an inference about an individual patient. A potentially large number of patient characteristics such as age, sex, co-morbid illnesses, physical findings, symptoms, disease history, and test results play an important role in determining precise assessments of risk and prognosis. For a radiologist, clinical decision making often concerns whether or not to image a patient, which imaging test to use, and making a diagnosis from the imaging findings. These decisions involve weighing the likelihood that the patient has the disease, the accuracy of the test for detecting the disease, potential harms that may result from the test, and the added value of the test in contributing to management. As a result, clinical decisions in radiology typically necessitate that health care providers perform difficult cognitive tasks. The following is a list of only some of the tasks involved in daily clinical care: • Assess the probability that a patient is diseased based on the patient’s characteristics • Choose among options for diagnostic testing, including sequences of imaging to further refine the probability of disease • Modify pre-test probability of disease based on diagnostic test results and estimate a post-test probability of disease • Estimate the risk of adverse effects of a therapy or imaging procedure • Quantify the probability that a given therapy will relieve disease symptoms or improve prognosis • Weigh the risk of adverse effects of therapy or imaging procedure against the potential benefit for each patient Examples from the research literature demonstrate that clinicians perform these cognitive tasks imperfectly.1,2,3,4 In addition to the challenges posed by the volume and complexity of information, physicians also need to understand patient values and help patients choose diagnostic and therapeutic options that align with their values. For example, while a CT scan may be the most accurate diagnostic test for investigating acute abdominal pain in a child, the parents may prefer ultrasound to avoid exposing the child to potentially unsafe radiation levels. In the subsequent sections we discuss the following reasons why clinical decisions are difficult: (1) lack of knowledge; (2) difficulties in risk estimation; (3) uncertainty; (4) heuristics and biases, in particular, the availability and representativeness heuristic and confirmatory bias; and (5) values. One of the biggest limitations in decision-making is insufficient knowledge. It is a significant challenge for clinicians to recall, stay abreast of, and appropriately apply the vast amounts of information available to inform medical practice. Clinicians absorb a vast amount of information during their early career training and new information related to clinical care is generated at a rapid pace. Furthermore, the clinical setting is characterized by a great degree of complexity. Important factors like the prevalence of disease vary widely for different populations as defined by age, sex, ethnicity, socioeconomic status, place of birth, and comorbid illness, and have an effect on the predictive values of diagnostic tests. Patients with the same disease may have different symptoms, and thus the need to remain vigilant due to the possibility of unique disease presentations increases complexity. In one experiment, physicians from Australia and England were presented with clinical scenarios for chest pain, leg swelling, and atrial fibrillation.1 Each was presented with a vignette describing patient characteristics, symptoms, and clinical findings. Based on patient features, physicians were asked to estimate pre-test probability of diseases including deep venous thrombosis and stroke—conditions where imaging would be used to clarify the diagnosis. Estimates varied widely: for the clinical scenarios with the best performance, only approximately 55% of physicians were within 20% of the correct risk estimate. Results like these have important implications for radiologists, since an inaccurate estimation of pretest probability for disease significantly impacts the predictive value of a diagnostic test. This further complicates the decision of whether or not a particular diagnostic test should be performed in a given clinical scenario by influencing the degree to which the test outcome adds value to patient management. Individuals have difficulties accurately estimating risks. Individuals tend to overestimate the risk of events with a low probability and underestimate the risk of events with a high probability.5 For example, individuals may be more concerned about the risk of developing cancer from exposure to low doses of radiation and less concerned about the risk of cancer associated with obesity. This distortion in risk perception may lead someone to be more vigilant about advocating for increased government controls on radiation emissions near a nuclear power plant and less vigilant maintaining a healthy weight and eating nutritiously. It is difficult for both clinicians and patients to think in probabilistic ways. For example, when doctors in Ontario were asked to estimate the 8-year probability of developing heart disease (angina, myocardial infarction, or coronary death) for hypothetical patients, they systematically overestimated each patient’s absolute risk.2 The physicians also overestimated the reduction in risk association with modifying one or more risk factors. Interestingly, both this and another study found that the physicians performed better when estimating the relative risk of disease due to the presence of an important risk factor than when estimating the baseline risk and the risk reductions for specific interventions.2,3 Only 24% of physicians had accurate risk estimates, but demonstrated greater accuracy in estimating the expected risk reduction from therapy, with 43% estimating relative risk accurately. The degree of overestimation was larger for low-risk scenarios, intermediate for medium-risk scenarios, and smaller for high-risk scenarios.3 A European study highlighted the difficulty faced by radiologists when attempting to quantify patients’ risk of developing contrast-induced nephropathy from exposure to contrast agents.4 The majority of radiologists surveyed used unstandardized and invalidated criteria for assessment of risk of contrast-induced nephropathy, which led to important inaccuracies in their risk estimates. Most radiologists underestimated the risk of contrast-induced nephropathy, and this may have significant clinical implications. The concept of uncertainty is distinct from risk, but equally important in clinical decision making. Han and colleagues define uncertainty as “the subjective perception of ignorance that is experienced by patients and health professionals in differing ways and degrees, motivates action, and elicits a variety of psychological responses.”6 In clinical decisions, there are two sources of uncertainty about probabilities or risks: (1) uncertainty about the exact value of a probability; and (2) uncertainty about what will happen in the future, even when there is certainty about the exact probability value. The latter type of uncertainty is easily recognizable. When tossing a coin, the probability of heads is known to be 0.5, but with any given toss the outcome is uncertain. Even if we accurately estimate a patient’s risk of developing cancer, unless the probability is 100%, we are still uncertain about what will actually happen to the patient. The former type of uncertainty, uncertainty about the exact probability value, arises largely from the complex factors involved in estimating risk and prognosis that have already been discussed. An honest assessment of the complexities involved in risk assessment and a thorough understanding of human cognitive limitations would lead to an appreciation of uncertainty. Evidence shows that clinicians tend to be overly confident that their risk estimates are correct. In the United Kingdom, nursing students and experienced nurses were presented with 25 vignettes and were asked to estimate the risks of adverse events.7 The vignettes outlined vital signs and consciousness levels for hypothetical patients. Both groups performed poorly at estimating the risk of an adverse event. Experienced nurses had greater confidence in their judgments than students, but there were no statistical differences in the rates of incorrect risk estimates. Clinicians overly confident about risk estimates may provide suboptimal guidance to patients. Patients may also be overconfident about their risk estimates. In diagnostic testing, patients’ preferences and preconceived notions about risks of imaging play an important role in decision making. A study performed in the pediatric setting highlighted the knowledge gap experienced by parents of children with head injuries regarding adverse events associated with radiation. Although approximately 50% of parents surveyed claimed to be aware of an association between CT imaging and potential cancer risk, over 90% of parents were initially willing to allow their children to undergo CT scanning of the brain; this number decreased to approximately 70% after discussion of potential long-term cancer risks.8 The work of Amos Tversky and Daniel Kahneman demonstrates that when individuals make decisions, they frequently rely on intuitive reasoning in place of more deliberative processes in order to derive predictions and arrive at judgments faster.9 These intuitive short cuts are known as heuristics and they can lead to systematic errors in judgment that are called biases.9 Use of heuristics and biases in judgment that results is not restricted to the lay public but has been demonstrated in educated people like clinicians with training in statistical analyses. Heuristics can lead to inaccurate estimates of probability and suboptimal clinical guidance. The availability heuristic, the representativeness heuristic, and confirmatory bias are three examples that can lead to errors in clinical judgment. Individuals rely on past experiences to predict future events. The availability heuristic manifests when an individual estimates the probability of future events based on easily remembered past events. A doctor who diagnoses a rare disease in a patient may place a higher probability on the presence of that disease in future similar patients. For example, a family physician who discovers a lump during routine breast examination of a 35-year-old woman that is eventually revealed to be cancerous may overestimate the likelihood of breast cancer in future young women presenting with a lump on breast examination. As a result, the doctor may refer many more young women to screening mammography than if that first 35-year-old woman did not have cancer. In other words, the discovery of a cancerous lesion in a young woman is a memorable event that leads the clinician’s focus away from hard data on the prevalence of disease in women with the same characteristics. The availability heuristic leads to accurate estimation of the probability of future events only if the memorable nature of past events equates to the frequency of the events, a situation that is rarely true. This heuristic contributes to limitations in clinical decision making by leading to inaccurate assessment of probability. The representativeness heuristic manifests when a clinician allows the degree to which a patient’s symptoms resemble a disease state to signify the likelihood that the patient has the disease. Dawson and Harkes explain representativeness heuristic as instances when “resemblance is used as a quick means of assessing likelihood.”10 For example, the authors cite the example of a young, healthy male presenting with pleuritic chest pain, hemoptysis, and shortness of breath.10 The symptoms were consistent with two conditions: pulmonary infarction and bacterial pneumonitis. Representativeness bias would lead a clinician to falsely conclude that since the symptoms were consistent with two conditions that each condition was equally possible. In fact, because the symptoms don’t provide sufficient information to allow a clinician to differentiate between the two possible diagnoses, the best indicator of the correct diagnosis is the prior probability of disease. Thus, bacterial pneumonitis would have been more likely based on its prevalence in young, healthy males. Individuals have the tendency to emphasize the importance of information that fits already formed hypotheses and de-emphasize information that contradicts these hypotheses.10 A radiologist may ignore negative or “normal” findings on an X-ray or CT scan when they contradict his/her hypothesis that disease is present.11 A radiologist may seek confirmatory evidence of a hypothesis and reinterpret imaging findings in such a way as to support a preexisting hypothesis.12 The weight of importance that individual patients place on aspects of clinical decisions increases the complexity of clinical decision making. One patient may accept high-risk procedures such as surgery or radiation-bearing imaging examinations if there is a greater probability of gain, while another patient may avoid risk and opt to cope with disease symptoms to avoid exposure to risks of surgery or radiation. Patient values and preferences are highly variable.13 Clinicians may not always elicit information from patients on their values and preferences, and thus may suggest courses of action inconsistent with patient values and preferences. Decision analysis has the potential to improve clinical decision making. Decision analysis can address and potentially overcome many of the limitations to clinical decisions identified above. By explicitly representing the features of a decision—the options, possible outcomes, associated probabilities, and values—decision analysis quantifies the relative desirability of each option based on these features. Using an explicit, quantitative framework, like that provided by decision analysis, creates the incentive to identify estimates from the best sources and specify accurate numerical risk quantities, overcoming lack of knowledge. Patient factors associated with risk, such as age, sex, and clinical findings, can be incorporated into a decision analysis with weights derived from validated risk algorithms to compute risk precisely. Indeed, validated risk algorithms can be used to calculate risk estimates based on patient characteristics. Decision analysis addresses problems of risk estimation through mathematical calculation, for example by combining the pretest probability of disease with the test result and accuracy of the test to calculate the posttest probability of disease. Inaccuracies due to heuristics and biases can be overcome by the quantitative, rule-based nature of decision analysis. Decision analysis addresses uncertainty by allowing the analyst to vary inputs within a plausible range and test changes in the desirability of each decision option. Patient values can be incorporated into a decision analysis to identify the best option. A decision tree is both a model of a decision that facilitates decision analysis, and also a tool that represents key decision features such as the decision options, possible events, probabilities, and outcomes.14 A decision tree is frequently represented graphically, and the decision tree features can be used to illustrate the rules associated with decision analysis. The decision tree has the following explicit components: • A list of options to choose from • Pathways to specify the sequence of events that can occur with each option • Probabilities or other parameters associated with each event • Outcomes representing the summation of a pathway • Values indicating the desirability of the outcomes The decision tree has the following implicit components: • An individual (or group) whose perspective is represented by the tree. The options, pathways, and values are relevant to the individual (or group) • A time frame that captures relevant events Each explicit component of a decision tree is typically represented graphically. While the implicit components are not represented graphically, they must be specified and understood to facilitate interpretation of the tree.
Decision Analysis
Learning Objectives
Concepts
Decision Analysis
What Makes Clinical Decisions Difficult?
Lack of Knowledge
Difficulties in Risk Estimation
Uncertainty
Heuristics and Biases
Availability Heuristic
Representativeness Heuristic
Confirmatory Bias
Values
How Can Decision Analysis Improve Clinical Decision Making?
Decision Tree
Radiology Key
Fastest Radiology Insight Engine
