American College of Radiology Appropriateness Criteria

By the early 1990s, the growth of healthcare costs had begun to draw serious attention from policymakers. Spending on physician services was a significant driver of this growth, and it was evident that expenditures for radiology services were increasing at a faster rate than expenditures for most other physician specialties.

In April 1993, then-chairman of the American College of Radiology (ACR) Board of Chancellors K.K. Wallace Jr, MD, testified before the House of Representatives Ways and Means Health Subcommittee regarding the Clinton administration’s proposed 1994 Medicare budget. Dr. Wallace stated that the ACR was prepared to define a system of patient care guidelines for radiology to address and eliminate inappropriate utilization of imaging services. He indicated that such guidelines could produce savings for the healthcare system without degrading the quality of care. His proposal formed the basis of what became the ACR Appropriateness Criteria (ACR-AC), evidence-based guidelines developed “to assist referring physicians and other providers in making the most appropriate imaging or treatment decision for a specific clinical condition.”

Prior to the development of the ACR-AC, the federal Agency for Healthcare Research and Quality (AHRQ, then known as the Agency for Health Care Policy and Research) published attributes for developing acceptable clinical practice guidelines designed by the Institute of Medicine. Major characteristics for acceptable guidelines include validity (guidelines should lead to better outcomes), reproducibility (another set of experts should be able to produce the same guidelines when using the same methodology and scientific evidence), and clinical applicability , among others. The ACR Task Force on Appropriateness Criteria incorporated these attributes into the development of the ACR-AC, and the ACR-AC topics are included in the material posted online by the National Guidelines Clearinghouse (NGC), a publicly available database of evidence-based clinical guidelines compiled by the AHRQ.

The Ambulatory Quality Alliance, a voluntary collaborative of several organizations including the AHRQ, has developed a set of principles to provide guidance for organizations developing appropriateness criteria. The ACR has embraced their definition of appropriateness: “The concept of appropriateness, as applied to healthcare, balances risk and benefit of a treatment, test, or procedure in the context of available resources for an individual patient with specific characteristics. Appropriateness criteria provide guidance to supplement the clinician’s judgment as to whether a patient is a reasonable candidate for the given treatment, test or procedure.”

Although scientific evidence is intended to be the basis for guidelines like the ACR-AC, the AHRQ acknowledges that judgment and group consensus are a necessary part of the development process. Because of this, development of the ACR-AC takes place under the guidance of expert panels, which for diagnostic imaging are organized along body systems. A large number of physicians volunteer their time for this effort, including nonradiologists representing over 20 medical specialty organizations. Funding and staff support to the panels are provided entirely by the ACR.

Panels are typically composed of more than 10 members, each of whom has expertise in the clinical condition for which the criteria are being developed. The chairs of these panels serve on the Committee on Diagnostic Imaging/Interventional Radiology Appropriateness Criteria, which oversees the activities of all the subspecialty panels. A separate body, the Committee on Radiation Oncology Appropriateness Criteria, performs the same function for radiation oncology topics.

In addition to the expert panels, other committees also assist in the development and maintenance of the ACR-AC. The Subcommittee on Radiation Exposure assigns relative radiation levels for the procedures included in the AC topics and provides information about radiation dose assessment. The Subcommittee on Appropriateness Criteria Methodology monitors and reviews the methods used in developing and revising the AC topics; it is composed of members from the various expert panels, as well as others with special expertise in relevant fields such as research methodology.

The development of AC topics by the panels is a rigorous process based on a systematic review of available scientific evidence. Topic selection may be based on disease prevalence and morbidity, variability of current practice, cost considerations, and potential for improved care. Topics are intended to be clinically salient and commonly encountered in daily practice. Examples include such entities as “acute chest pain,” “headache,” and “blunt abdominal trauma.” Once a topic is chosen, the clinical conditions to be addressed are defined as specifically as possible, sometimes resulting in a number of variants within each topic. Variants usually reflect a clinical scenario encountered within the topic. For example, the “headache” topic includes such variants as “chronic headache” and “sudden onset of severe headache.” With guidance from the panels, ACR staff then perform a search of the peer-reviewed medical literature and compile articles relevant to the condition being studied. The principal topic author then drafts a narrative , complete with citations, which summarizes the available evidence in the literature. ACR staff further assist by creating an evidence table , which rates the study quality for each article cited in the narrative. Both the narrative and evidence table are reviewed by members of the expert panel, and the author uses their feedback to modify these documents.

After the narrative and evidence table for a topic have been modified and final versions sent to the expert panel, panel members rate the appropriateness of each diagnostic procedure or treatment for each variant. Each member assigns a rating to each procedure based on his or her interpretation of the evidence and personal assessment of the risks and benefits of performing the procedure. Note that when scientific evidence for a specific topic or variant is uncertain or lacking, expert opinion may serve as the primary basis for determinations of appropriateness by panel members.

The process of assigning appropriateness ratings is based on the RAND/UCLA Appropriateness Method (developed by the RAND Corporation and the University of California, Los Angeles) and uses a structured communication technique known as a modified Delphi method. This method allows rater anonymity during the process and is believed to improve the chances of achieving consensus. Appropriateness ratings are assigned on an ordinal scale using integers from 1 to 9, grouped into three categories. Ratings of 1 to 3 are categorized as “usually not appropriate,” categories 4 to 6 are designated as “may be appropriate,” and categories 7 to 9 are termed “usually appropriate.” The “may be appropriate” category results when the evidence is contradictory or unclear, risks and benefits are equivocal or uncertain, there is significant inter-rater disagreement, or special circumstances may influence the risks or benefits of the procedure in question.

An example of the published appropriateness ratings for the clinical condition Suspected Spine Trauma is provided in Table 29.1 . In the first variant, a clinical situation in which a patient meets the low-risk criteria for cervical spine imaging by the commonly used National Emergency X-Radiography Utilization Study (NEXUS) criteria or Canadian C-Spine Rule, all of the listed procedures are assigned the lowest possible rating of 1, indicating that ordering imaging studies is usually not appropriate. In the second variant, in which a patient meets clinical criteria for imaging, cervical spine computed tomography (CT) without contrast is assigned the highest possible rating of 9. These ratings make clear that CT, and not radiographs, is the preferred imaging modality in high-risk cases of cervical spine trauma. Note that the Relative Radiation Level for each procedure is provided in the last column.

As of March 2016, the ACR-AC include numerous topics in diagnostic radiology, interventional radiology, and radiation oncology, covering a total of 211 clinical conditions with more than 1000 variants. Diagnostic imaging topics are sorted into 10 subspecialty areas: Breast, Cardiac, Gastrointestinal, Musculoskeletal, Neurologic, Pediatric, Thoracic, Urologic, Vascular, and Women’s. Because new scientific evidence about the efficacy and value of imaging modalities for different conditions is continually being introduced, the ACR-AC are frequently reviewed and updated by the expert panels. Topic revisions ideally occur approximately every 3 years.

No matter how expansive or comprehensive they become, of course, the ACR-AC cannot assist in patient care unless referring physicians and other clinicians are aware of their existence and use them to make clinical decisions. Low awareness of the ACR-AC is a problem with roots in the medical education and training process: one study showed that an overwhelming majority (96%) of third- and fourth-year medical students were unaware of the AC as an available resource. Knowledge about the AC also lags among residents in nonradiology specialties, with multiple studies demonstrating poor resident performance when asked to answer multiple-choice questions based on AC topics.

Scarce medical literature has addressed the extent of incorporation of the ACR-AC into clinical practice. A 2009 study revealed exceptionally low utilization of the AC among both residents and attending physicians responding to an online survey, with only 2.4% of 126 respondents reporting using the AC when ordering studies (it should be noted, however, that the study excluded emergency physicians). When respondents were asked which resource served as their first choice when choosing the most appropriate imaging study for patients, the ACR-AC ranked last behind several other options, including Google searches and a panoply of other online resources.

Concerns about poor awareness and knowledge of the ACR-AC have led to several educational efforts targeted at medical students and residents based on the premise that improved awareness of the AC among trainees will lead to improved utilization. Educational interventions taking place as early as during the preclinical curriculum of medical school have shown promise. In a 2013 study, second-year medical students participated in a 3-day radiology elective focused on the ACR-AC, comparative effectiveness in imaging, and the risks of radiation from medical imaging. At the conclusion of this radiology elective, the students demonstrated improvement in their knowledge of appropriate utilization of medical imaging and perceived awareness of the indications, contraindications, and effects of radiation exposure from medical imaging when compared to peers who had participated in a different elective. Participants in the radiology course also reported feeling more confident in their ability to order appropriate imaging studies when investigating common emergency complaints. Without long-term data, however, it remains uncertain if this kind of curricular change can have a meaningful effect on the ordering behavior of budding physicians once they are practicing independently.

Most of the studies addressing the use of the ACR-AC in resident education have examined educational programs within radiology residencies. Some of these studies have been able to show improved awareness and/or knowledge regarding the AC among radiology residents after exposure to material on imaging appropriateness. Unfortunately, little data exist demonstrating a link between this increased awareness and outcomes for patients and the healthcare system. The only study assessing the broader impact of resident education in this area showed that after participating in lectures about cost-consciousness and hospital charges for abdominal imaging, internal medicine residents ordered fewer abdominal CT and radiographic examinations per patient. This resulted in a cost reduction of $129 per patient. If these results can be replicated on a larger scale and prove to be durable over time, the ACR-AC may be able to fulfill their promise of reducing imaging overutilization and delivering cost savings to the healthcare system.

The ACR-AC also have some limitations. First of all, the appropriateness of imaging in a given scenario basically addresses the initial imaging encounter with the patient and does not discuss further testing as the evaluation progresses, as might be experienced with a diagnostic algorithm or decision-tree approach. Second, although many topics and variants are discussed, the ACR-AC cannot be considered exhaustive in the larger context of all of the possible clinical dilemmas encountered in a busy clinical practice. Third, because the topics are periodically revised and never retired, the committee has limited capacity for introducing new topics. Finally, given the lengthy process of revising topics, rapidly moving developments in a given field may not be reflected in the ACR-AC in a timely fashion.

The ACR is not the only prominent medical specialty organization working to promote evidence-based utilization of imaging services. The American College of Cardiology Appropriate Use Criteria Task Force, in conjunction with several other specialty societies, has published several reports providing AUC for cardiovascular imaging. Topics addressed include cardiac radionuclide imaging, echocardiography, and peripheral vascular ultrasound. Additionally, in 2015 this American College of Cardiology Task Force and the ACR jointly published a major guidelines document in the Journal of the American College of Cardiology addressing 20 common clinical scenarios in which patients present to an emergency department with chest pain.

Finally, the efforts of medical organizations in other countries to promote imaging appropriateness bear mentioning. The Cochrane Library, maintained by a global network of collaborators based in the United Kingdom, is a collection of databases that are widely used as high-quality sources of scientific evidence to inform healthcare decision making. The Royal College of Radiology, the British counterpart to the ACR, produces evidence-based radiology referral guidelines similar to the ACR-AC. These guidelines, known as iRefer, are freely available online and as a smartphone application for practitioners in the UK’s National Health Service. The Canadian Association of Radiologists also publishes referral guidelines for diagnostic imaging, last updated in 2012.