Multimodality image registration and fusion have a key role in routine diagnosis, staging, restaging, and the assessment of response to treatment, surgery, and radiotherapy planning of malignant disease. The complementarity between anatomic (CT and MR imaging) and molecular (SPECT and PET) imaging modalities is well established and the role of fusion imaging widely recognized as a central piece of the general tree of clinical decision making. Moreover, dual modality imaging technologies including SPECT/CT, PET/CT, and, in the future, PET/MR imaging, now represent the leading component of contemporary health care institutions. This article discusses recent advances in clinical multimodality imaging, the role of correlative fusion imaging in a clinical setting, and future opportunities and challenges facing the adoption of multimodality imaging.
Medical imaging has evolved rapidly during the last 2 decades, and we are now observing radical changes in the way medicine is practiced as a logical consequence of this growth. Nowadays, clinical diagnosis is rarely done without imaging, which makes molecular imaging an essential component of the clinical decision-making tree. Contemporary molecular imaging technologies now represent the leading component of any health care institution and have a pivotal role in the daily clinical management of patients.
X-ray projection imaging, ultrasonography, CT, and MR imaging differentiate disease from normal tissue by revealing structural differences or differences in regional perfusion of the administered contrast media. The interpretation of the images can be complicated when normal perfusion patterns are disrupted by prior surgery or radiotherapy, which can lead to tissue damage or necrosis where contrast patterns can mimic those associated with neoplasia. This effect presents a significant challenge when imaging techniques are used to define the anatomic extent of disease, such as for planning highly conformal radiation treatment or highly targeted therapeutic regimens.
In comparison with anatomic imaging techniques, functional imaging methods including planar scintigraphy, single-photon emission computed tomography (SPECT), positron emission tomography (PET), and MR spectroscopy assess regional differences in the biochemical status of tissues. In nuclear medicine, including SPECT and PET, this assessment is done by administering a biologically active molecule or pharmaceutical to the patient which is radiolabeled and accumulated in response to its biochemical attributes. The realization that the information provided by anatomic (CT and MR) and molecular (SPECT and PET) imaging modalities is complementarity spurred the development of various strategies for multimodality image registration and fusion. Correlative or fusion functional-anatomic imaging is now well established and its clinical value widely recognized.
Several investigators proposed and in most cases developed techniques to improve the correlation between the anatomic and physiologic information obtained using these anatomic and functional imaging studies. These methods include software-based image registration in which two or more sets of images from two or more different studies are fused following their separate acquisition on stand-alone imaging systems. Commonly, image registration techniques produce a single “fused” or “combined” image in which the functional SPECT or PET image is displayed in color over a gray-scale CT or MR image of the same anatomic region. Alternatively, hardware-based, dual-modality imaging systems including SPECT/CT, PET/CT, and, in the future, PET/MR imaging, more successfully achieve this goal, which underlies their wider clinical acceptance by the medical imaging community.
This article discusses recent advances in clinical multimodality imaging and the role of correlative fusion imaging in the clinical setting. Future opportunities and challenges facing the adoption of multimodality imaging are also addressed.
Software-based image registration and fusion
Software image fusion can be challenging to perform on a routine basis in the clinical setting because it requires exceptional digital communication in medicine (DICOM) connectivity, compatibility between the scanning protocols used by various imaging modalities, and outstanding collaboration between various clinical departments. These challenges may be overcome by the use of combined PET/CT systems described in the following section, although software-based coregistration offers greater flexibility and might in some cases offer some complementary advantages to hardware-based approaches.
Achieving a high degree of accuracy for a spatial transformation between image sets can be complicated. Physical factors such as noise, limited spatial resolution, attenuation, scatter, and partial volume effect (PVE) and biologic factors such as persistent activity in the blood pool and nonspecific uptake may decrease the contrast and blur the images; therefore, it can be difficult to locate consistent landmarks. The coregistration problem in the brain is different from the situation in whole-body imaging. Furthermore, diagnostic CT images are usually taken using breath-holding techniques, whereas PET data are acquired during a relatively long time period with the resultant reconstructed image set being an average of all phases of respiration. PET/CT investigations involving imaging of the thorax, abdomen, or pelvis, where organ motion exists, result in inconsistent image sets. This inconsistency can cause complications, for example, if the body boundaries of the CT data and the PET can be registered but the internal structures still differ significantly. Various PET/CT scanning protocols performed for a short period but with a similar breathing pattern have been designed to avoid the breath-holding problem. The CT data acquired allow for both attenuation correction and registration of PET/CT data for accurate localization of metabolic abnormalities. Despite their difficulties, many semi- or fully automated registration methods have been developed and used with various degrees of success in research and clinical settings. An in-depth overview of software-based registration techniques and algorithms is beyond the scope of this review. For a detailed survey of the algorithms developed so far, the reader is referred to recent comprehensive reviews.
Two main strategies have emerged in the literature to perform so-called “rigid registration,” such as brain PET-MR imaging registration of images of the same patient. The first strategy is based on the identification of similar structures in both images and subsequent minimization of a “distance measure” between them. The second strategy uses a voxel-per-voxel similarity measure of the full three-dimensional data set as a matching criterion (where voxel stands for a volume element , ie, a three-dimensional image point). The criterion that drives the registration algorithm is known as the “similarity measure.” The most popular similarity measures find their origin in information theoretic approaches. These approaches include minimization of histogram dispersion, maximization of mutual information, or maximization of the correlation ratio. The most widely used criterion is mutual information, an intensity-based similarity measure, and many variants to this approach (eg, normalized mutual information) have subsequently been proposed in the literature. Nonrigid registration approaches are usually required to correlate images of the thorax and abdomen. These approaches are usually combined with linear registration techniques to correct for changes in body configuration, differences in breathing patterns, or internal organ motion and associated displacements. Within the context of the assessment of response to treatment in which intrapatient registration of pre- and post-treatment whole-body PET images may be required to automate the analysis of lesion size and uptake, nonrigid registration with position-dependent rigidity approaches have been suggested. These techniques assign a high degree of rigidity to some regions (eg, lesions, brain) that will remain unchanged following the registration process.
Hardware-based multimodality imaging
Combined PET/CT Instrumentation
The historical development of multimodality imaging is marked by various significant technical and scientific accomplishments driven by an unprecedented collaboration between multidisciplinary groups of investigators. Even though the introduction of commercial PET/CT units in a clinical setting is a recent feature, the prospective benefits of correlative multimodality imaging have been well established since the early years of medical imaging. Many pioneering radiologic scientists and physicians recognized that the capabilities of a radionuclide imaging system could be improved by adding an external source to allow acquisition of transmission data for anatomic correlation of the emission image. Interestingly, the derived theoretical concepts that were occasionally patented never materialized in practice until the late Dr Bruce Hasegawa and colleagues at the University of California, San Francisco pioneered in the 1990s the development of dedicated SPECT/CT. Dr Hasegawa is the person to credit for the conception and design of the first combined SPECT/CT unit, which now stands as a wonderful tribute to his memory. Later, Dr Townsend and coworkers at the University of Pittsburgh pioneered in 1998 the development of combined PET/CT imaging systems, which have the capability to record both PET emission and x-ray transmission data for correlated functional/structural imaging. More compact and cost-effective designs of dual-modality systems have been explored more recently. One such approach uses a rail-with-sliding-bed design in which a sliding CT bed is placed on a track in the floor and linked to a flexible SPECT camera. A variety of rail-based, docking, and click-over concepts for correlating functional and anatomic images are also being considered with the goal of offering a more economic approach to multimodality imaging for institutions with limited resources.
Among the many advantages offered by PET/CT is a reduction in the overall scanning time, allowing one to increase patient throughput by approximately 30% owing to the use of fast CT-based attenuation correction when compared with lengthy procedures involving the use of external transmission rod sources. Fig. 1 illustrates the timeline for various stand-alone PET and combined PET/CT scanning protocols following tracer injection and the typical 1-hour waiting time for 18 F-fluorodeoxyglucose (FDG). The patient is prepared for imaging by administering the radiopharmaceutical, typically 370 to 555 MBq (10 to 15 mCi) of 18 F-FDG in adults. A pre-injection transmission scan is usually performed on stand-alone PET scanners before tracer injection to reduce spillover of emission data into the transmission energy window, although post-injection transmission scanning protocols have been successfully used in the clinic with the use of contemporary PET scanners. When using combined PET/CT units, the patient is asked to remove all metal objects that could introduce artifacts in the CT scan and is then positioned on the table of the dual-modality imaging system. The patient undergoes an “overview” or “scout” scan during which x-ray projection data are obtained from the patient to identify the axial extent of the CT and PET study. The patient then undergoes a low-dose spiral CT acquisition followed by the PET study starting approximately 1 hour after FDG administration. The CT and PET data are reconstructed and registered, with the CT data used for attenuation correction of the reconstructed PET images. Depending on institutions and agreements between clinical departments and clinical requirements, the images might be interpreted in tandem by a radiologist and nuclear medicine physician who can view the CT scan, the PET images, and the fused PET/CT data, followed by preparation of the associated clinical report. Some clinical indications commonly require administration with contrast media to acquire a relatively high-dose diagnostic quality CT scan. The latter scan can be performed either before or following the PET study. In the former case, the contrast-enhanced CT is also used to correct the PET data for photon attenuation, and the low-dose CT scan is no longer needed. Care should be taken to avoid hot-spot artifacts in the attenuation-corrected PET images that might be caused by overcorrection of radiodense oral and intravenous contrast agents. As a rule of thumb, examination of the uncorrected images is recommended to distinguish technical artifacts from physiologic/pathologic hypermetabolism. Alternatively, post-processing correction methods have been proposed in the literature.
Combined PET/MR Imaging Instrumentation
The interest in PET scanning within strong magnetic fields was first motivated by the need to reduce the distance positrons travel before annihilation (positron range) through magnetic confinement of the emitted positrons. Indeed, Monte Carlo simulation studies predicted improvements in spatial resolution for high-energy positron emitters ranging between 18.5% (2.73 mm instead of 3.35 mm) for 68 Ga and 26.8% (2.68 mm instead of 3.66 mm) for 82 Rb for a magnetic field strength of 7 T. These improvements are in agreement with the results obtained using another Monte Carlo code in which a 27% improvement in spatial resolution for a PET scanner incorporating a 10 T magnetic field was reported.
The history of combined PET/MR imaging dates back to the mid-1990s even before the advent of PET/CT. Early attempts to design MR-compatible PET units relied on slight modification of PET detector blocks of a preclinical PET scanner to keep the photomultiplier tubes (PMTs) at a reasonable distance from the strong magnetic field of a clinical MR imaging unit. The detectors were coupled to long optical fibers (4–5 m), leading the weak scintillation light outside the fringe magnetic field to position-sensitive PMTs. Despite the limitations of this design, similar approaches were adopted by other investigators. Other related design concepts based on conventional PMT-based PET detectors rely on more complex magnet designs, including a split magnet or field-cycled MR imaging.
Other investigators have developed PET/MR imaging systems configured with suitable solid-state detectors that can be operated within a magnetic field for PET imaging. These systems include avalanche photodiodes (APDs) and Geiger-mode avalanche photodiodes (G-APDs). APD-based readout has already been implemented on a commercial preclinical PET system, the LabPET scanner, 10 years after the development of the first prototype based on this technology. Various MR-compatible preclinical PET prototypes have been designed using both APD-based and G-APD based technologies. Other promising technologies that might be used for the design of future generation PET/MR imaging systems include amorphous selenium avalanche photodetectors, which have an excellent quantum efficiency, a large avalanche gain, and a rapid response time.
Most of these systems have been tested within a high field (up to 9.7 T) and have produced PET and MR images that appear to be free of distortion, consolidating the hypothesis that there is no significant interference between the two systems, and that each modality is virtually invisible to the other.
The promising results obtained on preclinical systems have encouraged one of the major industrial players (Siemens Medical Solutions, Knoxville, TN) to develop the first clinical PET/MR imaging prototype (BrainPET), dedicated for simultaneous brain imaging, in collaboration with the University of Tuebingen in Germany. Fig. 2 illustrates the conceptual design and a photograph of the integrated MR/PET scanner, showing isocentric layering of the MR head coil, PET detector ring, and MR magnet tunnel together with concurrently acquired clinical MR, PET, and fused MR/PET images. The system is being assessed in a clinical setting by exploiting the full potential of anatomic MR imaging in terms of high soft-tissue contrast sensitivity in addition to the many other possibilities offered by this modality, including blood oxygenation level dependant (BOLD) imaging, functional MR imaging, diffusion-weighted imaging, perfusion-weighted imaging, and diffusion tensor imaging. The prospective applications of a hypothetical whole-body PET/MR imaging system are being explored in the literature. Such a system would allow one to exploit, in addition to the previously discussed applications, the power of MR spectroscopy to measure the regional biochemical content and to assess the metabolic status or the presence of neoplasia and other diseases in specific tissue areas.
Hardware-based multimodality imaging
Combined PET/CT Instrumentation
The historical development of multimodality imaging is marked by various significant technical and scientific accomplishments driven by an unprecedented collaboration between multidisciplinary groups of investigators. Even though the introduction of commercial PET/CT units in a clinical setting is a recent feature, the prospective benefits of correlative multimodality imaging have been well established since the early years of medical imaging. Many pioneering radiologic scientists and physicians recognized that the capabilities of a radionuclide imaging system could be improved by adding an external source to allow acquisition of transmission data for anatomic correlation of the emission image. Interestingly, the derived theoretical concepts that were occasionally patented never materialized in practice until the late Dr Bruce Hasegawa and colleagues at the University of California, San Francisco pioneered in the 1990s the development of dedicated SPECT/CT. Dr Hasegawa is the person to credit for the conception and design of the first combined SPECT/CT unit, which now stands as a wonderful tribute to his memory. Later, Dr Townsend and coworkers at the University of Pittsburgh pioneered in 1998 the development of combined PET/CT imaging systems, which have the capability to record both PET emission and x-ray transmission data for correlated functional/structural imaging. More compact and cost-effective designs of dual-modality systems have been explored more recently. One such approach uses a rail-with-sliding-bed design in which a sliding CT bed is placed on a track in the floor and linked to a flexible SPECT camera. A variety of rail-based, docking, and click-over concepts for correlating functional and anatomic images are also being considered with the goal of offering a more economic approach to multimodality imaging for institutions with limited resources.
Among the many advantages offered by PET/CT is a reduction in the overall scanning time, allowing one to increase patient throughput by approximately 30% owing to the use of fast CT-based attenuation correction when compared with lengthy procedures involving the use of external transmission rod sources. Fig. 1 illustrates the timeline for various stand-alone PET and combined PET/CT scanning protocols following tracer injection and the typical 1-hour waiting time for 18 F-fluorodeoxyglucose (FDG). The patient is prepared for imaging by administering the radiopharmaceutical, typically 370 to 555 MBq (10 to 15 mCi) of 18 F-FDG in adults. A pre-injection transmission scan is usually performed on stand-alone PET scanners before tracer injection to reduce spillover of emission data into the transmission energy window, although post-injection transmission scanning protocols have been successfully used in the clinic with the use of contemporary PET scanners. When using combined PET/CT units, the patient is asked to remove all metal objects that could introduce artifacts in the CT scan and is then positioned on the table of the dual-modality imaging system. The patient undergoes an “overview” or “scout” scan during which x-ray projection data are obtained from the patient to identify the axial extent of the CT and PET study. The patient then undergoes a low-dose spiral CT acquisition followed by the PET study starting approximately 1 hour after FDG administration. The CT and PET data are reconstructed and registered, with the CT data used for attenuation correction of the reconstructed PET images. Depending on institutions and agreements between clinical departments and clinical requirements, the images might be interpreted in tandem by a radiologist and nuclear medicine physician who can view the CT scan, the PET images, and the fused PET/CT data, followed by preparation of the associated clinical report. Some clinical indications commonly require administration with contrast media to acquire a relatively high-dose diagnostic quality CT scan. The latter scan can be performed either before or following the PET study. In the former case, the contrast-enhanced CT is also used to correct the PET data for photon attenuation, and the low-dose CT scan is no longer needed. Care should be taken to avoid hot-spot artifacts in the attenuation-corrected PET images that might be caused by overcorrection of radiodense oral and intravenous contrast agents. As a rule of thumb, examination of the uncorrected images is recommended to distinguish technical artifacts from physiologic/pathologic hypermetabolism. Alternatively, post-processing correction methods have been proposed in the literature.
Combined PET/MR Imaging Instrumentation
The interest in PET scanning within strong magnetic fields was first motivated by the need to reduce the distance positrons travel before annihilation (positron range) through magnetic confinement of the emitted positrons. Indeed, Monte Carlo simulation studies predicted improvements in spatial resolution for high-energy positron emitters ranging between 18.5% (2.73 mm instead of 3.35 mm) for 68 Ga and 26.8% (2.68 mm instead of 3.66 mm) for 82 Rb for a magnetic field strength of 7 T. These improvements are in agreement with the results obtained using another Monte Carlo code in which a 27% improvement in spatial resolution for a PET scanner incorporating a 10 T magnetic field was reported.
The history of combined PET/MR imaging dates back to the mid-1990s even before the advent of PET/CT. Early attempts to design MR-compatible PET units relied on slight modification of PET detector blocks of a preclinical PET scanner to keep the photomultiplier tubes (PMTs) at a reasonable distance from the strong magnetic field of a clinical MR imaging unit. The detectors were coupled to long optical fibers (4–5 m), leading the weak scintillation light outside the fringe magnetic field to position-sensitive PMTs. Despite the limitations of this design, similar approaches were adopted by other investigators. Other related design concepts based on conventional PMT-based PET detectors rely on more complex magnet designs, including a split magnet or field-cycled MR imaging.
Other investigators have developed PET/MR imaging systems configured with suitable solid-state detectors that can be operated within a magnetic field for PET imaging. These systems include avalanche photodiodes (APDs) and Geiger-mode avalanche photodiodes (G-APDs). APD-based readout has already been implemented on a commercial preclinical PET system, the LabPET scanner, 10 years after the development of the first prototype based on this technology. Various MR-compatible preclinical PET prototypes have been designed using both APD-based and G-APD based technologies. Other promising technologies that might be used for the design of future generation PET/MR imaging systems include amorphous selenium avalanche photodetectors, which have an excellent quantum efficiency, a large avalanche gain, and a rapid response time.
Most of these systems have been tested within a high field (up to 9.7 T) and have produced PET and MR images that appear to be free of distortion, consolidating the hypothesis that there is no significant interference between the two systems, and that each modality is virtually invisible to the other.
The promising results obtained on preclinical systems have encouraged one of the major industrial players (Siemens Medical Solutions, Knoxville, TN) to develop the first clinical PET/MR imaging prototype (BrainPET), dedicated for simultaneous brain imaging, in collaboration with the University of Tuebingen in Germany. Fig. 2 illustrates the conceptual design and a photograph of the integrated MR/PET scanner, showing isocentric layering of the MR head coil, PET detector ring, and MR magnet tunnel together with concurrently acquired clinical MR, PET, and fused MR/PET images. The system is being assessed in a clinical setting by exploiting the full potential of anatomic MR imaging in terms of high soft-tissue contrast sensitivity in addition to the many other possibilities offered by this modality, including blood oxygenation level dependant (BOLD) imaging, functional MR imaging, diffusion-weighted imaging, perfusion-weighted imaging, and diffusion tensor imaging. The prospective applications of a hypothetical whole-body PET/MR imaging system are being explored in the literature. Such a system would allow one to exploit, in addition to the previously discussed applications, the power of MR spectroscopy to measure the regional biochemical content and to assess the metabolic status or the presence of neoplasia and other diseases in specific tissue areas.
Clinical role of correlative fusion imaging
The clinical role of correlative imaging encompasses a wide variety of applications. It is now performed routinely with commercially available radiopharmaceuticals to answer important clinical questions in oncology, cardiology, neurology, and psychiatry. As discussed previously, much of the early image registration effort was restricted to intrasubject brain applications, where the confinement of compact brain tissues within the skull renders a rigid-body model a satisfactory approximation. Correlative fusion imaging techniques were introduced in the clinic, mostly for neuroimaging applications, well before the advent of hardware-based, dual-modality imaging. Multimodality imaging had a pivotal role in the assessment of central nervous system disorders such as seizures, Alzheimer’s and Parkinson’s disease, head injury, and inoperable brain tumors.
Brain SPECT imaging using 99m Tc-labeled perfusion ligands shows a sharp increase during an epileptic seizure (ictal scan) at the position of the epileptogenic focus, whereas most epileptic foci show a diminished perfusion on the interictal scan. By means of ictal/interictal subtraction studies, with subsequent coregistration onto MR imaging ( S ubtraction I ctal S PECT Co registered to M R imaging [SISCOM]), a predictive value up to 97% for the correct localization of an epileptic focus has been reported, which is higher than any other competing modality. Fig. 3 shows an example of a 99m Tc-labelled ethylene cysteine dimer (ECD) perfusion SPECT and FDG-PET studies of the same patient coregistered to an anatomic T1-weighted MR imaging study for the evaluation of epilepsy. The two 99m Tc-ECD scans were performed during seizure (ictal) and when the patient was seizure free (interictal) the following day. Both SPECT studies and a three-dimensional, T1-weighted MR imaging study were coregistered using the normalized mutual information criterion, which is similar to mutual information but usually more robust and efficient in finding the correct fitting transform. The differences between the ictal and interictal SPECT studies were overlaid on transaxial slices of the MR imaging study to permit accurate localization of the focus of the epilepsy. A coregistered FDG-PET study superimposed on the MR imaging study is also shown. This type of image registration and fusion technique has been a standard component of many clinical practices for the last 2 decades and is used routinely in the authors’ institution. Corresponding techniques for other regions of the body have not achieved the same widespread clinical use.
Related posts:
Origins of Intraoperative MRI
MR Imaging–Guided Interventions in the Genitourinary Tract: An Evolving Concept
Magnetic Resonance Nephrourography: Current and Developing Techniques
MRI of Hip Osteoarthritis and Implications for Surgery
MRI in Dementia
Imaging of Lymphoma of the Musculoskeletal System
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
