The organization of the sensory surfaces is reflected even at the level of the cerebral cortex (Fig. 9.4). For example, in vision, separate representations of the two eyes are maintained through the LGN and into V1, forming ocular dominance columns that can be visualized in humans using fMRI (Cheng et al. 2001; Yacoub et al. 2007). Further, many regions in the visual cortex exhibit retinotopic organization with adjacent neurons responding to stimuli presented at adjacent locations in the visual field and thus stimulating adjacent points on the retina. Finally, the spatial sampling at the retina is much greater at the fovea than in the periphery, and this is reflected in a disproportionately large area of the cortex devoted to foveal vision compared to peripheral vision (Sereno et al. 1995).

Prior to fMRI, most of the knowledge of retinotopic maps in the brain was based on studies of human patients with lesions to the visual cortex (Holmes 1931; Horton and Hoyt 1991) and invasive studies in nonhuman primates (Tootell et al. 1982). However, fMRI has enabled the identification of a multitude of such retinotopic maps both in subcortical structures (LGN, Schneider et al. 2004; e.g., superior colliculus, Katyal et al. 2010) and throughout visual cortex from V1 onwards (e.g., Sereno et al. 1995). The presence of a complete map of visual space is one of the criteria used to define a distinct cortical area (for review, see Wandell et al. 2007).

Topographic maps can be identified by systematically varying specific stimulus dimensions and measuring the corresponding changes in the BOLD (blood oxygenation level dependent) response (Engel 2012). Typically, the dimensions manipulated reflect the dimensions mapped by the sensory surface. For example, visual field maps can be identified by systematically varying either the eccentricity (distance from the center of gaze) or polar angle (angular distance from the horizontal or vertical meridian) of a flickering checkerboard and estimating each voxel’s preferred visual field position along both dimensions (Engel et al. 1994; for review, see Wandell and Winawer 2011). An alternative model-based approach estimates the population receptive field for each voxel (Dumoulin and Wandell 2008). Using these methods, field maps have been identified not only in the occipital lobe (corresponding to V1, V2, and V3) (e.g., Sereno et al. 1995) but also more anteriorly along both the dorsal (e.g., Swisher et al. 2007) and ventral pathways (e.g., Larsson and Heeger 2006; Arcaro et al. 2009).

In audition, mapping studies have systematically varied the frequency of tones revealing tonotopic (or “cochleotopic”) maps, reflecting the organization of hair cells in the cochlea (Vonbekesy 1949). Such mappings reveal a mirror-symmetric tonotopic organization (high-low-low-high) in the human primary auditory cortex (Woods et al. 2009; Da Costa et al. 2011; Langers and van Dijk 2012) that runs across or perpendicular to Heschl’s gyrus (but see Formisano et al. 2003; Da Costa et al. 2011). Additional frequency gradients have been found in multiple sites in the auditory cortex (Talavage et al. 2000, 2004; Humphries et al. 2010), extending to higher-order auditory regions in the temporal lobe (Striem-Amit et al. 2011).

Perhaps the most renowned cortical maps are the somatotopic maps forming a somatosensory homunculus in the cortex. Consistent with Penfield’s groundbreaking electrical stimulation studies (Penfield and Boldrey 1937), human fMRI has revealed a highly detailed systematic representation of body surface along the postcentral gyrus and posterior bank of the central sulcus (Medina and Coslett 2010). The homunculus shows an upside-down medial-to-lateral organization in the primary somatosensory cortex: the feet, legs, and genitals are represented medially, whereas the face and hands are represented along the lateral aspects of the postcentral gyrus (Huang and Sereno 2007). The resolution of current fMRI methods is sufficient to distinguish between representations of the individual digits on the hand (Nelson and Chen 2008; Schweizer et al. 2008; Sanchez-Panchuelo et al. 2010; Stringer et al. 2011), with an ordered mapping from the thumb (lateral) to the little finger (medial) and identify distinct intradigit representations (Overduin and Servos 2004; Schweisfurth et al. 2011).

The overall medial-lateral topographic organization extends to the secondary somatosensory cortex (SII) in the parietal operculum, although the representation is less fine grained than in S1 (Disbrow et al. 2000; Ruben et al. 2001; Eickhoff et al. 2007).

The properties of these topographic maps are directly reflected in behavior. For example, in both somatosensory and visual processing, the size of the cortical representation in V1 or S1 for that location in visual space or for a given finger correlates with behavioral acuity (Duncan and Boynton 2003, 2007) and with the change in psychophysical thresholds following learning (Pleger et al. 2003). In turn, these cortical representations likely reflect both the density of receptors on the sensory surfaces and the density of connections between the thalamus and cortex.

In contrast to vision, audition, and somatosensation, the evidence for cortical topographic maps in the chemical senses (olfaction and gustation) is less clear. In olfaction, studies in rodents suggest a rough spatial organization at the level of the olfactory epithelium, with different olfactory receptors representing perceptually similar odorants clustered together that is also reflected in the organization of the olfactory bulb (for review, see Murthy 2011). This organization is not, however, retained in the cortex with olfactory bulb neurons projecting to very broad regions of the piriform cortex (Illig and Haberly 2003). Correspondingly, chemotopic organization has not been reported in human fMRI studies of the piriform cortex (Arzi and Sobel 2011). Nevertheless, fMRI has been used to uncover other forms of functional organization in the piriform cortex. For example, different parts of the piriform cortex are sensitive to different odorant attributes: the anterior piriform cortex is sensitive to the chemical structure of odorants, while the posterior piriform cortex is sensitive to the perceptual quality of odorants (Gottfried et al. 2006).

Finally, the human gustatory cortex has also not been found to contain systematic sensory maps. One major factor contributing to the lack of such an observation may be the large extent to which the gustatory cortex is multimodal and integrative (Small and Prescott 2005; Rolls 2006). fMRI studies suggest that besides encoding the basic tastes (bitter, sweet, salt, sour, umami: de Araujo et al. 2003), the gustatory cortex also responds preferentially to odors (Small et al. 2005), to the combination of odor and taste (Small et al. 2004), to somatosensory oral textures (e.g., viscosity and lubricity), and to their combination with particular tastes as well as other chemical properties, such as fat concentration (Eldeghaidy et al. 2011).

Beyond maps, which may reflect the organization of the sensory surfaces, neural representations in the cortex may cluster according to the properties or features of the stimuli themselves. This has been most extensively investigated in vision with specific visual areas or patches of cortex that are primarily responsive to features such as motion (Huk et al. 2002) or color (Conway et al. 2007; Wade et al. 2008).

Further, by contrasting the magnitude of response elicited during presentation of different categories of objects, fMRI has revealed a number of visual category-selective regions in the human occipitotemporal cortex (OTC) (Fig. 9.5, for review, see Kanwisher and Dilks 2013) that respond more strongly to one category (e.g., faces) than others (e.g., scenes). These include regions selective for faces (Puce et al. 1996; Kanwisher et al. 1997), objects (Malach et al. 1995; Kourtzi and Kanwisher 2000), letter strings or words (Puce et al. 1996; Cohen and Dehaene 2004; Baker et al. 2007), body parts (Downing et al. 2001), scenes (Aguirre et al. 1998; Epstein and Kanwisher 1998), animals (Chao et al. 1999), and tools (Chao et al. 1999). In many cases, there are at least two regions selective for a given category, one on lateral OTC and one on ventral OTC (Hasson et al. 2003; Taylor and Downing 2011). Support for a fundamental role of these regions in behavior comes from neuropsychological studies of patients with focal lesions in OTC producing specific impairments in the recognition of particular categories (faces, Barton and Cherkasova 2003; bodies, Moro et al. 2008; objects, Behrmann et al. 1992; scenes, Aguirre and D’Esposito 1999; words, Mycroft et al. 2009). Thus, while defining an object is a complex philosophical question, the striking clustering and functional specialization of neurons in OTC by object categories highlight the centrality of objects to our everyday experience (Martin 2009).

Strikingly, these different category-selective regions show a remarkable consistency in their topographical organization across individuals suggesting they may reflect some underlying principle of brain organization. For example, supported by the finding of category selective in the occipitotemporal cortex in the blind (Mahon et al. 2009), their locations may be indicative of cortical connectivity patterns between functionally related regions (Martin 2006; Mahon and Caramazza 2011). Similarly, their location may reflect retinotopic biases in high-level visual cortex. Specifically, the ventral surface of the temporal lobe shows an eccentricity bias with foveal stimuli biased to lateral regions and peripheral stimuli biased to more medial regions (Levy et al. 2001; Hasson et al. 2002). Since different categories and different tasks are associated with different goals requiring different types of visual information (Harel and Bentin 2009), such underlying principles of organization may constrain the location of category-selective regions. For example, given the necessity of foveating faces and words compared with scenes, the observed eccentricity biases may account for the lateral and medial locations of face-selective and scene-selective regions, respectively (Hasson et al. 2002). Consistent with this eccentricity bias, in which larger objects would stimulate more peripheral vision, a large-scale organization of visual representations according to real-world object size across the lateral and ventral surfaces has been reported (Konkle and Oliva 2012). Importantly, these principles are not mutually exclusive, and there may be multiple organizational principles underlying the observed pattern of responses (Op de Beeck et al. 2008).

While category appears to be a dominant organizing feature of high-level visual processing, it does not appear to be as dominant in the other modalities. In addition, many studies have emphasized distributed rather than localized representations of auditory categories (Formisano et al. 2008; Staeren et al. 2009).

There is some evidence for stronger responses to vocalizations, especially those of conspecifics, than other complex natural sounds in the auditory ventral pathway (Belin and Zatorre 2000; Lewis et al. 2005; Doehrmann et al. 2008; Talkington et al. 2012) with subregions selective for musical instrument sounds, human speech, and acoustic phonetic content (Leaver and Rauschecker 2010). However, the extent to which such distinctions simply reflect the spectrotemporal properties of the stimuli is unclear (Samson et al. 2011).

Most of the studies on higher-order representations discussed so far have focused on regional differences in the magnitude of response. While such an approach can readily provide evidence for gross distinctions in the cortex (e.g., face selectivity), it does not necessarily provide insight into the functional role of regions or the specific information being processed and may miss distinctions at a finer spatial scale.

The nature of representations within regions can be investigated by focusing on aspects of the fMRI response other than simple magnitude (see Boxes 9.2 and 9.3). For example, the scene-selective parahippocampal place area (PPA) on the ventral OTC responds more strongly to scenes than to other categories such as objects and faces (Fig. 9.5). Studies focusing on the specific pattern of response across this region using either hypothesis-driven or data-driven approaches have demonstrated that PPA contains information about both spatial layout (Fig. 9.6, Kravitz et al. 2011a; Park et al. 2011) and object identity (Harel et al. 2013), consistent with the inputs it receives from both the ventral and dorsal pathways (Kravitz et al. 2011b) and suggesting a key role in navigation.

In olfaction, while there is no evidence for distinct patches coding odor categories, the pattern of response in the posterior piriform cortex does reflect odor category, correlating with perceptual judgments of odor quality and similarity (Howard et al. 2009). The more similar odorants were perceived, the more similar the patterns of response elicited (Fig. 9.7).

Insights into cortical organization can also be revealed by searching the entire brain for the information contained within patterns of response, rather than just simply differences in response magnitude (Kriegeskorte et al. 2006). For example, studies searching for regions distinguishing between the faces of different individuals have highlighted the importance of anterior temporal lobe regions (Kriegeskorte et al. 2007; Nestor et al. 2011), even though face selectivity tends to be weak in this region. Similarly, a study searching for areas containing head-view-invariant representation of gaze direction identified regions in both the anterior and posterior superior temporal sulcus, although the posterior region was more dependent on the physical properties of the stimuli (Carlin et al. 2011, see Fig. 9.8).

Overall, there is a pronounced organization in high visual cortex (i.e., categories) that is not clearly evident in the other modalities. However, there are clearly high-order representations in other modalities, and any such organizational principles may occur at a spatial scale below the resolution of current fMRI.

Top-down attention involves enhancement (or prioritization) of particular aspects of incoming sensory information based on the specific goals of the observer. Attention can be directed to spatial locations (space-based attention) or features of the sensory input (e.g., object-based attention in vision). At a neural level, attention manifests in several different forms including lowering of firing thresholds, scaling of response magnitude, and enhanc synchronization (Reynolds and Chelazzi 2004; Lee and Maunsell 2009; Reynolds and Heeger 2009).

Effects of top-down attention have been reported with fMRI even as early as the subcortical structures. For example, the visual response in LGN is modulated by spatial attention with enhanced responses to attended visual stimuli and suppressed responses to ignored stimuli (O’Connor et al. 2002). Attention has also been shown to modulate visual responses in the pulvinar (Kastner et al. 2004) and superior colliculus (Schneider and Kastner 2005; Katyal et al. 2010). These effects of top-down input highlight that the subcortical structures are more than just passive relay structures, as has often been assumed (for review and further examples, see Saalmann and Kastner 2011).

Similar effects of attention have been reported in the early visual cortex including V1. Thus, directing attention to a particular spatial location results in superior task performance for stimuli appearing in that location, concomitant with an enhancement of the BOLD response in the corresponding retinotopic regions of the cortex (Tootell et al. 1998; Brefczynski and DeYoe 1999). Further, the attentional effects on the BOLD response are constrained by the receptive size of the regions along ventral visual pathway (Kastner et al. 1999; Muller et al. 2003).

How attention is allocated can be influenced by features of the visual stimuli and not just by space. For example, following a spatial cue on an object, retinotopic activity in early visual areas is enhanced for other locations on that same object, but not equally distant locations outside the object (Muller and Kleinschmidt 2003), a phenomenon known as object-based attention (see also Martinez et al. 2006; Ciaramitaro et al. 2011). Effects of attention to objects and features have also been demonstrated in higher-order category-selective regions. For example, attending to a particular object category (faces or houses) enhances activity in the respective face- or scene-selective regions (O’Craven et al. 1999). The top-down modulatory effect on category-selective regions has been suggested to originate in the prefrontal cortex and can be seen even before stimulus presentation in the form of increased activity with the expectation of incoming stimuli (for review, see Gazzaley and Nobre 2012). Such object-based attentional effects are thought to reflect increased neural firing rate for the attended stimulus, as well as increased selectivity for features of the attended stimulus (Murray and Wojciulik 2004).

Top-down modulations of the sensory cortex by attention, expectations, and task have also been observed with fMRI in modalities other than vision. For example, activity in the olfactory cortex (Fig. 9.9) was found to increase when participants sniffed non-odorant air but were expecting an odor (“attended sniff” condition) relative to when they sniffed the same non-odorant air but were not expecting an odor (“unattended sniff“ condition) (Zelano et al. 2005). Further, activity in part of the piriform cortex in the “attended sniff” condition was enhanced in response to auditory task instructions, before the stimulus was even presented, indicating a strong effect of expectation. Higher-order olfactory regions, such as the orbitofrontal cortex, are also modulated by odor content expectations (de Araujo et al. 2005), and attention to odors appears to modulate functional connectivity between the piriform and orbitofrontal cortex through the mediodorsal nucleus of the thalamus (Plailly et al. 2008).

Similar to the effects observed for olfaction, trying to detect taste in a tasteless solution results in enhanced activation in the middorsal insula and frontal operculum, corresponding to the primary gustatory cortex (Veldhuizen et al. 2007). Further, manipulation of expectation also modulates fMRI responses in the gustatory cortex (Nitschke et al. 2006; Small et al. 2008; Veldhuizen et al. 2011a). Importantly, these attentional effects in olfaction and gustation are modality specific (Veldhuizen and Small 2011).

In sum, top-down attentional effects can be seen across modalities and are one mechanism underlying the active, constructive process of perception, which utilizes our prior knowledge and experience of environmental regularities to form hypotheses and expectations about the external world.

Even in the absence of external sensory stimulation, it is still possible to generate internal representations using top-down signals, a process that is often referred to as mental imagery (Kosslyn et al. 2006). Such mental imagery can provide important insights into the nature of the top-down signals and their potential effect on perceptual processing along the sensory hierarchies. A general finding across modalities is that mental imagery elicits activity in similar cortical regions to those involved in perception (e.g., olfaction, Levy et al. 1999; somatosensory, Yoo et al. 2003; vision, Ganis et al. 2004; gustation, Kobayashi et al. 2004; audition, Bunzeck et al. 2005), and it is often suggested that perception and imagery share the same neural substrate. However, simply showing activation in the same regions is not sufficient to show that imagery and perception share similar mechanisms. Such activation could simply reflect a nonspecific engagement of a region. For example, in the case of olfaction, odor imagery is associated with sniffing, which is known to elicit activation on its own (Sobel et al. 1998). To demonstrate that activation of the olfactory cortex during imagery is odor specific rather than a general modulation, activations elicited by pleasant and unpleasant odors during both perception and imagery were compared, revealing similar hedonic specificity in both cases (Bensafi et al. 2007). In vision, studies have revealed category specificity in high-level visual areas during imagery (O’Craven and Kanwisher 2000; Reddy et al. 2010) and even decoding of individual objects (Lee et al. 2012). Furthermore, the specific representations elicited are similar during perception and imagery suggesting extensive overlap in the neural substrate of each (Stokes et al. 2009; Lee et al. 2012). Thus, imagery not only does engage the same regions as perception but can evoke highly specific representations (Fig. 9.10).

One area of contention is whether imagery-elicited activation extends all the way back to the primary sensory cortices. This debate has been entwined with the discussion of whether imagery is symbolic or depictive (Pylyshyn 2002; Kosslyn et al. 2006). In addition, it has been reported that only the secondary auditory cortices and not the primary auditory cortex is involved in auditory imagery (Bunzeck et al. 2005). However, in vision, retinotopically specific activity has been reported in the primary visual cortex (Slotnick et al. 2005; Thirion et al. 2006) and the activity in early visual cortex correlates with the vividness of imagery (Lee et al. 2012). In olfaction, the extent to which the primary olfactory cortex is active correlates with experience, with professional perfumers showing stronger activation than novices (Plailly et al. 2012).

So far we have highlighted the similarities between perception and imagery, but introspectively, they are very different phenomena. The strength of activation is clearly different between perception and imagery, but other differences have also been reported. For example, one study reported greater deactivation in the auditory cortex during visual imagery compared with visual perception, suggesting enhanced unimodal processing during imagery (Amedi et al. 2005). In another visual imagery study, the similarity of representations across different regions along the ventral visual pathway was greater during imagery than perception, suggesting different neural dynamics (Lee et al. 2012).

In sum, the study of mental imagery shows that top-down processes are able to elicit highly specific responses in sensory areas. The relationship between imagery and perception is complex, with overlap in the neural substrates involved but differences in the interregional relationships.

While the large-scale circuitry of the brain is present at birth, it is clear that there is the potential for extensive plasticity early in life and the nature of perceptual representations is dependent on experience throughout development. Even in primary sensory cortices, many of the properties of neurons and maps depend on experience early in life. For example, in V1, neural properties such as orientation selectivity and ocular dominance are dependent on early postnatal experience (Espinosa and Stryker 2012). Such changes are difficult to observe with fMRI due to the substantial practical challenges posed by scanning young children (but see, e.g., Arichi et al. 2010; Perani et al. 2010; Raschle et al. 2012). fMRI studies of visual development in infant monkeys have suggested that while there is strong activation in V1 early in life, there is only weak activation in extrastriate areas such as MT and V4 (Kourtzi et al. 2006). Studies in older children suggest a prolonged time course for developmental changes that may extend into adulthood. For example, in audition, a cross-sectional study of music perception in 5–33-year-olds revealed selective activation for melody and rhythm processing that increases over time even in the absence of any musical training (Ellis et al. 2012). In vision, much work has focused on high-level category-selective regions and in particular the face-selective FFA (fusiform face area). At a behavioral level, many of the markers of specialized face processing are present by 3–4 years of age (Sangrigoli and de Schonen 2004; Cassia et al. 2009), although performance improves through adolescence (e.g., Mondloch et al. 2003; for review, see McKone et al. 2012). Many fMRI studies report increases in size, response magnitude, or selectivity of FFA (e.g., Golarai et al. 2007; Scherf et al. 2007, 2011) with a smaller FFA in 12–16-year-olds than in adults (Golarai et al. 2010). However, it has been questioned whether methodological issues might account for such effects, rather than changes in neural processing (McKone et al. 2012). In particular, a major concern is the reduced signal strength observed when putting a child’s head inside an adult-sized head coil – it may be critical to match the size of head coil to the size of the child’s head (Keil et al. 2011).

One of the simplest forms of experience is prior exposure to a given perceptual stimulus. Such stimulus repetition leads both to decreased BOLD responses (variously termed repetition suppression or adaptation) throughout brain regions active in a given task, not just sensory areas, and also improvement in behavioral performance (termed repetition priming) (Grill-Spector et al. 2006). Such repetition suppression has been reported in multiple modalities and has been used as a way of probing neural representations with fMRI (see Box 9.2). The reduction in the BOLD response is immediate (Kourtzi and Kanwisher 2001), increases with multiple presentations (Sayres and Grill-Spector 2006), survives intervening stimuli (Vuilleumier et al. 2002; Weiner et al. 2010), and occurs even with delays of 3 days (van Turennout et al. 2000). However, the relationship between the decreased BOLD response, behavioral improvement, and the underlying neural mechanisms is unclear (Wiggs and Martin 1998; Henson 2003; Grill-Spector et al. 2006). How is it that reduced activity can be associated with enhanced performance? Proposals have included a more rapid neural response (James and Gauthier 2006), a sharper or more-selective response (Desimone 1996), or some suppression of bottom-up sensory responses by top-down predictions (Friston 2005). One intriguing suggestion is that the reduced neural activity is associated with enhanced neuronal synchronization producing more effective long-range communication between brain regions (Gotts et al. 2012).

Beyond repetition suppression, perceptual representations can change as a result of active training or learning. One way to investigate the effect of learning is to use a cross-sectional approach and compare subjects with varying amounts of perceptual expertise. Such studies have focused on differences in both functional and anatomical properties. For example, studies of musicians compared with nonmusicians have reported increased gray matter volume in the auditory cortex that correlates with musical aptitude (Schneider et al. 2002; Gaser and Schlaug 2003). Further, musical training correlates with stronger activation in the superior temporal gyrus (Koelsch et al. 2005) and in the planum temporale, just posterior to the primary auditory areas (Ohnishi et al. 2001). In gustation/olfaction, wine sommeliers show enhanced activation in the insula and orbitofrontal cortex (Castriota-Scanderbeg et al. 2005), and in somatosensation, Mah-Jong experts, who have learned tactile discrimination of the 2D shapes on Mah-Jong tiles, show enhanced activation of V1 (Saito et al. 2006) suggesting changes in cross-modal associations.

In the visual domain, cross-sectional studies have focused on expertise in the recognition of object categories, such as cars and birds. Faces can be thought of as a perceptual category we are all experts in processing (Diamond and Carey 1986), and it has been proposed that processing objects of expertise should engage the same neural structures that are engaged by faces (Tarr and Gauthier 2000).

While some studies have reported enhanced activation in FFA when car or bird experts view objects from their domain of expertise relative to control objects (Gauthier et al. 2000; Xu 2005), other studies failed to replicate these findings (Grill-Spector et al. 2004; Rhodes et al. 2004). Further, the enhanced activity for car experts viewing cars is both widespread, extending well beyond the FFA and even into parietal and prefrontal regions, and diminished when the experts are not engaged with their objects of expertise, suggesting that it may reflect enhanced attention (Harel et al. 2010). This same group of car experts also showed increased gray matter volume in the prefrontal cortex and not in the visual cortex (Gilaie-Dotan et al. 2012) suggesting that the neural changes involved in expert recognition are not necessarily perceptual and may involve top-down factors, such as knowledge and attention.

Since studies of real-world expertise are correlational in nature, it is impossible to avoid the “chicken and egg problem”: perhaps the specialized neural correlates exhibited by experts do not reflect the acquisition of expertise, but are in fact a precondition for the successful acquisition of expertise. Thus, an alternative approach is to provide laboratory training and measure the changes in perceptual representations directly. The advantage of such longitudinal studies is that they provide causal evidence for the relationship between training and neural changes. Across modalities, these studies have revealed changes in cortical responses following training, in the form of both increases and decreases. For example, in audition, frequency discrimination training over a week leads to reduced BOLD responses in the auditory cortex (Jancke et al. 2001). In contrast, odorant learning produces increased responses in the piriform and orbitofrontal cortex (Li et al. 2006). In vision, training on a simple visual texture discrimination task produces enhanced activity in the early visual cortex after only a single session of practice (Schwartz et al. 2002). However, both increases and decreases in visual responses across distributed regions have been reported for trained versus untrained shapes (Kourtzi et al. 2005), novel objects (“smoothies,” “spikies,” “cubies” Op de Beeck et al. 2006), and bird types (van der Linden et al. 2008). In the context of comparing object expertise to face expertise, some studies have reported changes in FFA following training on novel objects (“Greebles” Gauthier et al. 1999), while others have not (Op de Beeck et al. 2006; Wong et al. 2009), even with the same training paradigm (Brants et al. 2011). Although the presence of both increases and decreases in response following perceptual learning both within and across studies may seem contradictory, there are two major factors that need to be considered. First, the nature of the stimuli may determine which preexisting representations training modifies, by either strengthening or weakening. For example, whether responses in FFA change during training may depend on how face-like the stimuli are (Brants et al. 2011). Second, the nature of the task and the informativeness of any given neural population for that task will determine their likely modulation in learning (Op de Beeck and Baker 2010). Neural populations that are engaged by and used in performing a task will be more likely to show increased responsiveness.

Changes in functional response properties have also been reported following loss of sensory input. Complete loss of input in a given modality seems to produce some form of cross-modal plasticity (Merabet and Pascual-Leone 2010). For example, in the blind, a number of studies have demonstrated enhanced responsiveness to auditory (Gougoux et al. 2005; Poirier et al. 2006) and tactile (Burton et al. 2002) stimuli in visual areas (V1, MT), and others have reported activity even during verbal memory tasks (Amedi et al. 2003). Similarly, visual and somatosensory activation of the auditory cortex have been reported in the early deaf (Finney et al. 2001; Karns et al. 2012). However, these cross-modal effects are stronger with loss of input early in life and may reflect developmental plasticity mechanisms (Merabet and Pascual-Leone 2010). Further, such plasticity may be driven by enhancement of preexisting cross-modal responsiveness, which is normally masked by the dominant modality. Support for this “unmasking” comes from studies showing tactile responses in V1 of sighted subjects following blindfolding (Merabet et al. 2007, 2008). However, cross-modal responses in the early blind cannot be predicted from cross-modal responses in sighted subjects (Lewis et al. 2010), and it may be that unmasking of preexisting connections is accompanied by more permanent structural changes with long-term loss of sensory input (Merabet and Pascual-Leone 2010).