Key points

Transitions in translational imaging

There has been a huge increase in functional magnetic resonance imaging (fMRI) studies over the past 3 decades devoted to characterizing the neural architecture of memory processes, including encoding and retrieval of different types of material, identifying patterns that correlate with intraindividual and between-group variance in memory success, and parsing different types of memory operations such as retrieval of item versus relational information. The field is so rich in data and the taxonomy of memory terminology sufficiently well-defined that semiautomated meta-analyses of relevant data have been undertaken using techniques such as activation likelihood estimation. Furthermore, fMRI studies are now affording key insights about the merits of different memory theories in ways that were unimaginable just a decade ago.

Given the abundance of cognitive neuroscience data and techniques available to identify different associations and dissociations among brain regions and memory operations, it is reasonable to assume that the clinical application would also be well advanced. Certainly the rapid movement from the laboratory to the clinic for functional mapping of sensorimotor and language systems, where these data are used to inform neurosurgical decisions in a growing number of centers, might imply that memory would not be far behind. However, this is not yet the case; clinical applications of memory fMRI remain in the realm of research rather than being standard of care at present. Nonetheless, there are some successes to celebrate, some important lessons learned in trying to establish a strong evidentiary basis for the translation to clinical utility, and some exciting new paths forward that are discussed in this review. Throughout, examples are drawn mainly from the context of temporal lobe epilepsy (TLE), as it is in this context that many important advances have been made (see recent reviews ). However, data from other conditions including mild cognitive impairment (MCI), Alzheimer disease, and transient global amnesia are also helpful, as they illustrate key points with respect to interpretation of the fMRI blood oxygenation level dependent (BOLD) signal in memory processes.

Transitions in translational imaging

There has been a huge increase in functional magnetic resonance imaging (fMRI) studies over the past 3 decades devoted to characterizing the neural architecture of memory processes, including encoding and retrieval of different types of material, identifying patterns that correlate with intraindividual and between-group variance in memory success, and parsing different types of memory operations such as retrieval of item versus relational information. The field is so rich in data and the taxonomy of memory terminology sufficiently well-defined that semiautomated meta-analyses of relevant data have been undertaken using techniques such as activation likelihood estimation. Furthermore, fMRI studies are now affording key insights about the merits of different memory theories in ways that were unimaginable just a decade ago.

Given the abundance of cognitive neuroscience data and techniques available to identify different associations and dissociations among brain regions and memory operations, it is reasonable to assume that the clinical application would also be well advanced. Certainly the rapid movement from the laboratory to the clinic for functional mapping of sensorimotor and language systems, where these data are used to inform neurosurgical decisions in a growing number of centers, might imply that memory would not be far behind. However, this is not yet the case; clinical applications of memory fMRI remain in the realm of research rather than being standard of care at present. Nonetheless, there are some successes to celebrate, some important lessons learned in trying to establish a strong evidentiary basis for the translation to clinical utility, and some exciting new paths forward that are discussed in this review. Throughout, examples are drawn mainly from the context of temporal lobe epilepsy (TLE), as it is in this context that many important advances have been made (see recent reviews ). However, data from other conditions including mild cognitive impairment (MCI), Alzheimer disease, and transient global amnesia are also helpful, as they illustrate key points with respect to interpretation of the fMRI blood oxygenation level dependent (BOLD) signal in memory processes.

The goals of clinical functional MR imaging for memory

It is important to start by considering the goals of memory fMRI memory in clinical contexts. In patients with recurrent seizures arising from one temporal lobe, the concern is whether its removal in an effort to eliminate seizures will come at a significant cost in the form of postsurgical memory decline. It is well established that the functional adequacy of epileptogenic tissue in the medial temporal lobe (MTL) is a risk factor for postoperative change, in that better preoperative performance is associated with greater decline. Neuropsychological testing reveals that generally the left MTL is most involved in verbal memory functions whereas the right mediates visuospatial memory, and that information is used to inform both diagnosis (ie, does the memory impairment align with structural magnetic resonance [MR] imaging and electroencephalographic evidence) and prediction (ie, how much of a loss might be expected in a particular memory domain). Nonetheless, there are some weaknesses in the material specificity alignment, which undermine strong inferences about functional adequacy of MTL to be resected; these may be attributed to variation in processing strategies, test sensitivity, or possible reorganization of memory systems in epilepsy. In principle, fMRI is uniquely suited to investigate the functional anatomy of memory, and is thus poised to contribute a more direct metric of functional adequacy of the epileptogenic tissue. With a robust metric relating functional activation to clinical memory performance, one might predict more confidently whether an individual patient is likely to suffer a clinically significant decline. Leaving aside the issue of how best to characterize such a decline (eg, a statistically significant drop in memory test scores, subjective concerns, evidence of impairments in everyday functioning), the aim of identifying fMRI metrics of functional integrity has generated considerable research in epilepsy surgery centers.

In addition to epilepsy surgery, there are other clinical contexts in which determining the functional integrity of memory-relevant regions is important. The hippocampus and entorhinal cortex have long been recognized as structures showing early damage in the pathologic progression to Alzheimer disease, and measurement of hippocampal volume and entorhinal cortex thickness are now foundational elements of protocols for diagnosis and staging progression in MCI and Alzheimer disease. As described later, recent fMR imaging studies of memory in that population have begun to reveal interesting patterns of activation in these MTL structures that may well provide another useful biomarker of disease progression and may even illuminate the fundamental biological mechanisms of that progression. Furthermore, although there are relatively few studies at present, the use of memory activation tasks to assess the impact of rehabilitation strategies (whether cognitive, pharmacologic, or behavioral) on regional functioning and as an index of plasticity offers enormous promise.

The pragmatics of clinical functional MRI for memory

The procedures for imaging in clinical contexts involve several choices. As outlined in Box 1 , there are many decision points throughout the investigation and there are no simple plug-and-play solutions for imaging memory. The choices begin with selecting an activation task that is likely to be sufficiently reliable, sensitive, and readily implemented in a clinical context. Although there are elegant examples of activation tasks that demonstrate robust MTL activity in healthy young adults, they often require many repetitions (to accumulate sufficient data to achieve adequate signal-to-noise ratio), leading to unacceptably long imaging times (eg, 30 minutes or longer) for ease of clinical implementation. In addition, it is important to consider whether a task has proved to be sensitive to individual differences and to correlate with clinical features of memory, because deriving meaningful cutoffs for what will be interpreted as adequate, as opposed to defective activation, will necessitate proving its relationship to outcome data. This aspect is exemplified herein with the studies on TLE.

In identifying focal activation, there is nothing in the “raw” BOLD signal that is informative, so analysis typically requires contrasts between conditions, such as memory versus baseline or items subsequently recalled versus items subsequently forgotten. Having then determined the statistical threshold to use (too stringent and nothing shows as activated, too liberal and most of the brain might appear engaged), the next decision is the metric to be used to characterize functional integrity. In epilepsy, one could look at asymmetry of activation between MTL regions in each hemisphere, which is best if it can be expected that the normative patterns for both hemispheres show the same magnitude and/or extent of activation under the probe task. If there is no expectation of symmetry, comparison against some other standard is required, which may rely on a considerable amount of normative data on the same task over many individuals who are matched for important features (including age) with the epilepsy group, to ascertain whether the degree of activation in the affected MTL appears to be outside of the normal range. As discussed later, both strategies have been used in epilepsy. Validation of the clinical utility of any measure requires comparing it with appropriate neuropsychological measures of current memory performance and/or with scores that reflect presurgical to postsurgical changes on those measures. However, the field is not yet sufficiently mature that clinical cutoffs have been derived, and very few studies have looked at sensitivity and specificity of the fMRI indicator. The general rubric of poor activation signaling a defective neural substrate and, thus, less overall risk is one that has general consensus but, as discussed later, bigger is not invariably better in the sense of greater functional integrity.

Memory in temporal lobe epilepsy: activation tasks and correspondence with clinical memory data

A survey of the cognitive neuroscience literature indicates that tasks that reliably activate the MTL in healthy individuals share a few characteristics. Those that rely on encoding of complex novel visual information, such as scenes, in comparison with baseline repeated images are strong activators of MTL, including posterior regions such as the parahippocampal gyrus. Tasks that involve relational processing (eg, pairs of elements such as word-word or word-face; item-context associations) typically engage this region to a greater extent than those focusing on single elements, at both encoding and retrieval. Finally, very robust activation of the MTL bilaterally is typically observed when individuals retrieve memories of events from their own personal past (ie, details of autobiographical memories) in comparison with other forms of semantic retrieval (typically word meaning but also facts about one’s life). Of importance, the latter 2 distinctions (relational and autobiographical) also have strong behavioral evidence for the necessity of hippocampal involvement, as such processes are impaired in patients with medial TLE (mTLE) or following unilateral excision of a temporal lobe.

This review concentrates on studies in which fMRI results were correlated with or used to predict performance on neuropsychological memory tests in patients with mTLE. Scene encoding was one of the earlier paradigms used, likely because of the robust bilateral MTL activation generated by the contrast of complex novel scenes against a baseline (either “scrambled” picture or the same scene presented multiple times), the simple instructions (“try to remember the new scenes”), and the statistical power of a block-design presentation (alternating 20–30-second blocks of novel and baseline conditions), thus lending itself to relatively short scanning times (approximately 5 minutes) ( Fig. 1 ). In the earliest study of patients with unilateral mTLE, Detre and colleagues showed that asymmetry of activation in a region of interest (ROI) including posterior hippocampus, parahippocampal gyrus, and fusiform gyrus corresponded to memory results of the sodium amobarbital inactivation procedure, such that the spatial extent of activation was smaller on the side with worse memory performance. Subsequent studies using the same paradigm extended these results, indicating that both asymmetry ratios and absolute activation within the epileptogenic hippocampus were correlated with postoperative memory outcomes. A similar result regarding asymmetric activation and its relation to memory performance has been shown for a task involving mental navigation in a familiar environment, namely, imagining walking through your home town. These studies indicate that the left or right MTL found to be compromised on clinical memory tests generates a weaker BOLD response during a memory challenge.

Scene-encoding fMR imaging. Regions in color reflect areas of greater activation for novel scenes versus a repeated scene. ( A ) Average of 10 healthy controls, threshold P <.001 (corrected for multiple comparisons by False Discovery Rate). ( B, C ) Individual patients with right medial temporal lobe epilepsy (mTLE), at a similar threshold. Note that it is difficult to discern differences between patients and normative control data, with respect to spatial extent or magnitude of activation, based on visual inspection.

Another strategy has been to use material-specific memory probes for fMR imaging that are more likely to preferentially activate one or the other mesial temporal region (eg, words for the left hemisphere, abstract designs for the right hemisphere). In this case one can investigate the functional adequacy of the epileptogenic MTL as described earlier, but also the possibility that the contralateral MTL has augmented processing for nonpreferred material, and thus whether there is evidence of neural plasticity attributable to recurrent seizures. A caveat is that this material specificity does not always map onto hemispheric asymmetry in fMRI signals, even in healthy individuals. A word-recollection paradigm was found in several studies to demonstrate that the degree of activation in the left MTL in left mTLE patients predicted the magnitude of postoperative verbal memory decline. However, another group using a different verbal memory task demonstrated instead a somewhat complex pattern of activation changes in multiple brain regions relative to controls that seems to undermine the simple functional adequacy approach. Some studies have examined multiple tasks expected to be more selective to left (words) or right (pictures, faces) MTL engagement, in an attempt to see whether correlation with material-specific declines in memory performance can be enhanced by this strategy. Several of these have demonstrated greater activation in the nonaffected relative to the epileptogenic MTL, in addition to material-specific differences between controls and patients (eg, greater right hemisphere activation during word encoding in left mTLE patients). There was support for using either activation asymmetry or absolute magnitude of activation in the epileptogenic MTL as an index of functional adequacy, in that BOLD magnitude therein correlated positively with material-specific memory preoperatively and also with the magnitude of postoperative decline. One study suggests that this pattern is characteristic of signals from the anterior MTL, with posterior MTL activation showing the converse pattern of a positive correlation between preoperative activation and postoperative performance.

The preoperative imaging data in these studies typically show greater activation in the contralateral MTL during memory tasks, but it is not clear as to whether this is a marker of adaptive plasticity or attempted compensation. Most of these studies are underpowered (eg, comprising 7–15 participants per group) to adequately address factors related to plasticity such as age of onset and duration of seizures. Furthermore, greater atypical contralateral (eg, right MTL for verbal material) activation in several studies has been shown to be associated with poorer memory for that type of material, suggesting that it is not truly adaptive or compensatory. In addition, one study reported that increased activation in the posterior MTL following anterior temporal lobe resection in patients with left mTLE was actually correlated with worse postoperative verbal memory performance. The question of whether “more is better” in terms of activation magnitude is discussed again later.

Although much of the energy in this clinical memory imaging has been devoted to identifying optimal memory tasks for activating the MTL and, thus, serving as a biomarker for functional integrity, several investigators have shown that the degree of left hemisphere activation during language comprehension or production tasks can be a powerful predictor of verbal memory decline following left anterior temporal lobe resection. Indeed, a direct comparison of a language lateralization measure and hippocampal asymmetry on a scene-encoding task showed that the former was a better predictor of verbal memory decline following dominant resections. There are several possible explanations for this result. It may be critical to look at activation in regions beyond the MTL to enhance prediction, as memory is undoubtedly supported by wider brain networks. In addition, the full capacity of the MTL and extended memory architecture may not be characterized best by activation tasks whereby there is likely to be considerable variability in performance adequacy or effort. Both of these themes are now explored in greater depth.

Challenges to the link between activation magnitude and functional integrity

Much of the foregoing interpretations are predicated on the expectation that a greater magnitude of activation signifies a more functionally intact anatomic substrate. It is a reasonable assumption, but one that may be incomplete or incorrect, particularly when activation is taking place in damaged regions of brain, as in the hippocampus of a patient with mesial temporal sclerosis (MTS). Furthermore, even in the healthy brain, the degree of MTL activation may reflect a combination of objective encoding and retrieval of details, and a subjective experience of recollection that may be divorced from reality under certain conditions. The author’s own experience with a memory task that produces very robust activation in the MTL, autobiographical recall, is a good object lesson regarding the complex relationship between memory-induced functional activation and clinical indicators. Retrieval of episodes from the personal past is well established as generating reliable activation in mesial temporal regions ( Fig. 2 ) and patients with TLE demonstrate marked deficits in recalling details of autobiographical experiences. Furthermore, mesial temporal damage in these patients results in a markedly reduced activation in the affected region and in the whole network of regions typically engaged during autobiographical recall. Nonetheless, the author’s group has found relatively poor correspondence between the magnitude or extent of hippocampal activation and performance on clinical memory tests, and even some negative correlations in certain tasks (McAndrews, unpublished data). Thus, one can have a task that clearly engages the MTL during scanning and is dependent on the MTL for execution, and yet not be able to identify a clear fMRI biomarker of integrity (see also Ref. ).

Autobiographical memory. Regions in color reflect areas of greater activation when recalling specific events from one’s own life experience versus sentence completion. ( A ) Average of 12 healthy controls, threshold P <.001 (corrected for multiple comparisons by False Discovery Rate). ( B ) Single healthy control, threshold P <.005 (uncorrected). ( C ) Patient with left mTLE, threshold P <.01 (uncorrected). Note that despite a more liberal statistical threshold, activation is much weaker and more restricted in the patient and is absent in both MTL regions despite a focal left epileptogenic abnormality.

One possibility is that there is a nonmonotonic function relating activation levels to clinical impairment in TLE. In comparison with controls, mTLE patients with good postscanning recognition showed increased bilateral MTL activation during an encoding task, whereas those with poor subsequent memory showed reduced activation. This same pattern has been observed in another disorder involving MTL damage, MCI. Sperling and colleagues observed that, compared with controls, patients with very mild memory deficits show hyperactivation in the MTL, whereas patients with more significant deficits show hypoactivation. This pattern could suggest an attempt at compensation early in the course of neurodegeneration, which then fails following a critical level of neuronal damage. However, there are also data suggesting that this hyperactivation in MCI may actually reflect a pathologic process, as individuals with greater activation at baseline show more pronounced cognitive decline several years later, and reducing hippocampal excitability pharmacologically has been shown to reduce activity and improve memory in MCI patients on a memory task sensitive to hippocampal integrity. Finally, the author’s group had the opportunity to scan scene encoding and recognition with an individual during and after an episode of transient global amnesia (TGA). This female patient failed to show MTL activation for the standard contrasts (novel scenes > fixation at encoding; old scenes > new scenes at recognition). However, she showed substantial bilateral hippocampal activation when all scenes were contrasted (old and new) against the fixation stimulus (a blurred repeating scene) during the recognition run; this was not seen in any controls or in the patient when she returned for scanning 3 months later when her memory had returned to normal. Although this result confirmed that the hippocampal tissue was viable in the sense that it was capable of generating a BOLD signal, it was clearly not an indicator of good memory during the TGA episode. These findings illustrate a critical point, namely that variations in engagement or strategies during an fMRI memory task may add a great deal of noise to the simple calculation that activation magnitude equals memory competence.