For two objects to be distinguished on the retina, light rays from the objects must define at least a minimum angle (the visual angle) as they pass through the optical center of the eye. The visual angle, expressed in units of minutes of arc, defines the visual acuity of the eye. An individual with excellent vision and who is functioning in ideal viewing conditions can achieve a visual angle of about 0.5 minutes of arc, which is close to the theoretical minimum defined by the packing density of cones on the retina [7].

The eye is extremely sensitive to small amounts of light. Although the cones do not respond to illumination levels below a threshold of about 0.001 candela/m², the rods are much more sensitive and respond to just a few photons. For example, as few as 10 photons can generate a visual stimulus in an area of the retina where the rods are present at high concentration [8].

Differences in signal intensity that can just be detected by the human observer are known as just noticeable differences (JNDs). This concept applies to any type of signal, including light, that can be sensed by the observer. The smallest difference in signal that can be detected depends on the magnitude of the signal. For example, the brightness difference between one and two candles may be discernible, whereas the difference between 100 and 101 candles probably cannot be distinguished. This observation was quantified by the work of Ernst Heinrich Weber, who demonstrated that the JND is directly proportional to the intensity of the signal. This finding was quantified by Gustav Theodor Fechner as

(5.1)

where I is the intensity of stimulus, dS is an increment of perception (termed a limen), and k is a scaling factor. The integral form of this expression is known as the Weber–Fechner law:

(5.2)

The Weber–Fechner law is similar to the expression for the intensity of sound in decibels, and it provides a connection between the objective measurement of sound intensity and the subjective impression of loudness [9].

As noted earlier, the human visual system cannot instantly form a high-resolution picture of the visual scenery as the high-definition foveal vision has a very limited angular coverage of 1–2°. The only exception is when the displayed scene is limited to the fovea such as in retinal projection displays. In normal viewing, however, this limited angular extent necessitates the visual search process by which the high-resolution foveal vision is moved from place to place across the image being interpreted. Up to now, visual search studies have not revealed any a priori patterns that can be generalized across different individuals. However, the following conclusions have been drawn:

1. The foveal vision covers only a fraction of the “real-estate” of the image, emphasizing the role of peripheral vision in the interpretation process.
   
2. Peripheral vision in global impression contributes to the subsequent foveal search process.
   
3. Global impression is very quick (less than a second).
   
4. Different interpreters have different search patterns, but, by and large, the experts have a more streamlined and efficient patterns making them able to identify the abnormalities quickly without extensive scanning of the entire image.
   
5. Although limited in details, there are general search patterns that are quite often characteristic of particular search tasks.
   
6. Foveal search is composed of fixations on specific points within the image. Fixations longer than about 0.3 seconds are believed to indicate some level of visual processing, even cognitive recognition, of specific image features. The fixation time tends to be shorter for true positive abnormalities than false positive ones. True negative interpretations tend to be associated with fixations of relatively short duration. The time signatures associated with false negatives tend to fall between those of true positives and true negatives.

Expertise plays a definite role in the interpretation process, echoing the fact that radiology education by and large involves development of interpretation expertise that provides for accurate medical image interpretation.

Expertise is defined as the ability to efficiently use contextual knowledge toward accurate and consistent diagnosis. In the context of image interpretation, expertise is the ability to perform perceptual and cognitive analysis of image features to render the diagnosis accurately. The hallmarks of radiology expertise are a rich structured knowledge of normalcy and “perturbations,” an efficient hypothesis-driven search strategy, and the ability to generalize visual findings to idealized patterns.

Referencing the four perceptual steps noted above, different interpretation errors occur corresponding to the stage of interpretation. Errors are equally attributed to visual and cognitive elements. About half of the errors (55%) tend to be visual in nature, reflective of recognition errors. They come from the incompleteness of the visual search, influenced by perceptual as well as psychological factors. Among those, in about half of the cases (30%), the interpreter fails to look at the lesion location with high-resolution foveal vision. In the other half (25%), the interpreter does not fixate on the location for at least a dwell time of approximately 0.3 seconds (this time signature is somewhat task dependent).

The other component of the errors is cognitive, reflective of decision errors. In about half of the cases of interpretation errors (45%), the interpreter’s fixation time is greater than approximately 0.3 seconds. In other words, the abnormality is duly visualized. However, the interpreter makes a decision-making error in recognizing its importance calling the case negative.

In addition to the detection errors of the visual or cognitive nature, there are also classification errors, calling a nodule benign when it is malignant or calling a pneumonia atelectasis. These types of errors are less extensively studied.

Image quality can have a profound effect on the interpretation process. In that regard, three components of the image quality can be recognized: inherent attributes, presentation attributes, and anatomical noise. The inherent attributes of image quality include factors such as contrast, sharpness, and noise, as discussed in Chapter 3. Depending on the indication and the imaging modality, clinicians expect a certain level of image resolution and noise for confident interpretation. Additional levels of blur or noise (possibly because of image degradation processes or low exposure) is believed to lead to low accuracy, equivocal decisions, inconsistent decisions, or lack of efficiency to be able to reliably render a diagnosis.

In addition to resolution and noise, the way an image is rendered via post-processing (see Chapter 3) and displayed can influence the image quality. Again, different modalities and indications have different norms in terms of what is clinically expected for a given interpretation. For example, mammograms are generally expected to be presented at a display contrast significantly higher than that for a chest radiograph. Deviation from that expectation can lead to interpretation error and variability.

The signal-to-noise model describes the ability of an observer to detect simple visual signals embedded in a noisy background. Noise in the image (e.g. quantum mottle, extraneous structures, and receptor artifacts) is referred to as external noise [15]. The model characterizes an ideal observer who detects signals with a certain likelihood whenever their amplitude differs from background by some prescribed threshold, usually a multiplicative factor of 2–5. The inability of a real observer to detect visual signals as well as the ideal observer is said to be caused by internal noise in the real observer. Sources of internal noise include distraction of the observer, preconditioning of the observer by the presence or absence of signals in previously viewed images, reduced intrinsic sensitivity to visual signals in some observers, and a myriad of other possible causes.

The performances of ideal and actual observers are compared in the margin for the simple case in which actual observers are asked to state whether a simple signal is present or absent in an image. The index of detectability on the ordinate (y-axis) is the inverse of the size of subtle lesions that can be detected with reasonable assurance. As shown in the illustration, the performance of the human observer usually falls short of that for the ideal observer. This difference may be due in part to selection by the human observer of a visual threshold different from that assumed for the ideal observer. It may also be caused by internal noise (unpredictability) in the observer.

Wagner and Brown have suggested that the performance of an imaging system can be characterized quantitatively by describing the ability of an ideal observer to detect simple visual signals provided by the system [16]. They have proposed this approach for evaluation and comparison of imaging systems used in clinical radiology.

The computational model of visual detection has evolved from experience in designing computer programs to extract information from images. The model proposes that the first step in detecting visual information involves the rapid identification of brightness (density) changes in the image, together with properties such as orientation, location, depth, and spatial frequency of these changes. This “primal sketch” of the image is obtained by a simple psychophysical reaction to the image without the use of a priori knowledge of the structure and function of the object represented in the image. That is, visual signals are extracted from the image without any assumptions about the nature of the image and the type of visual scene portrayed [17]. Initial applications of the computational model were promising; however, it has largely been revisited [18] by more current understanding of psychophysical properties of human vision as delineated below.

In this model of visual detection, the human visual system is characterized as a series of parallel vision channels, with a specific frequency range of visual information assigned to each channel. Several studies suggest that the frequency range of each channel is on the order of ±1 octave of spatial frequency information and that the visual system as a whole functions essentially as a frequency analyzer [18–20].

Many different types of observer models have been devised, some based on achieving the best possible classification accuracy given the statistics of the image data (e.g. the Bayesian ideal observer), whereas other models attempt to be more anthropomorphic (e.g. mimic the human visual system). The models also differ in how much prior information they have about the size, shape, and locations of the signals to be detected and also the noise statistics of the images. This section will focus on the simplest situation, the signal-known-exactly (SKE) paradigm. In this paradigm, the observer knows the size, shape, contrast, and location of the signal to be detected. Given an image, the observer is tasked to decide if the signal is present or not.

All observer models can be mathematically described as some operator that transforms the image data, g, into a scalar test statistic, λ, in a similar manner as described in the section on the receiver operating characteristic (ROC) analysis (see Chapter 3). Here, g is vector containing all the image pixels (i.e. for an N × N image or region of interest, g is an N² × 1 vector). The model then compares λ to a threshold, τ, to make a final binary decision (if λ > τ, then the signal is present, otherwise the signal is absent). As such, observer models lend themselves to ROC analysis, and the area under the curve (AUC) of the observer model is often used as the final image quality figure of merit. Alternatively, consider the two distributions of the test statistic under signal-present and signal-absent conditions. Let , , , and be the means and variances of λ under signal-present and signal-absent conditions, respectively. Another common figure of merit is the detectability index (also called d-prime) d′, which is defined as

(5.3)

Essentially, d′ is a scalar measure of the amount of separation (or overlap) between the decision variable’s distributions under each condition, normalized by the average variance of the distributions. Note that d′ is specific to the imaging task and to a given observer model.

The most well-known observer model is the Bayesian ideal observer. This model uses all available statistical information about the image to make its decision. Specifically, the ideal observer uses the likelihood ratio, L(g), as its test statistic. This ratio compares two conditional probabilities:

(5.4)

It can be shown that using the likelihood ratio (or any monotonic function thereof) maximizes the classification accuracy, and thus, the ideal observer represents the upper limit of detectability given the statistics of the images. However, to compute the test statistic for the ideal observer, one needs to know both the signal-present and signal-absent probability density functions. For real-world medical images, these high-dimensional functions are not known, can be difficult to empirically estimate, and are often highly nonlinear. Further, it has been shown that human observers are not ideal, and thus, the detection accuracy derived from the ideal observer will always outperform what would be expected of human observers. Traditionally, the ideal observer has been used in designing and optimizing the hardware/geometry of an imaging system in order to maximize the amount of information that is output by the imaging system. However, optimizing the clinical use of a system is better done with anthropomorphic observer models that better simulate the way humans perceive images.

One such model, which has been shown to correlate strongly with human observers, is the non-prewhitening (NPW)-matched filter observer. This model belongs to a family of models known as linear observers, meaning that they form their test statistic as a linear combination of the image data with a vector of template weights. Using linear algebra/matrix multiplication notation, the test statistic, λ for linear observers is computed as the inner product of the template, ω, with the image data, g, over a image of N pixels, as

(5.5)

Linear observer models differ in the template used, but all follow this equation to form their test statistic. The NPW observer uses the expected (i.e. average) difference between the signal-present and signal-absent images as its template, ω_NPW = ∆g = E[g_SP − g_SA]. Here, g_SP and g_SA are random vectors representing the signal-present and signal-absent images, respectively, E[] is the expectation operator. The “NPW” part of the model refers to the fact that the model does not attempt to account for noise correlations.

It turns out that for a linear shift invariant (LSI) system with wide-sense stationary noise statistics, the detectability index for a NPW model can be calculated in the spatial frequency domain as a function of the system noise power spectrum (NPS) and modulation transfer function (MTF) as

(5.6)

where u and v are spatial frequencies in the x and y directions, respectively, and W(u, v) is the task function (i.e. the Fourier transform of the signal to be detected). This can be directly extended to higher dimensional images as well. The NPW observer model can be extended to include an eye filter, E(u, v), that describes the frequency response of the human visual system. This modified model observer is called the non-prewhitening matched filter with eye filter (NPWE) observer. The d′ for the NPWE observer is given as

(5.7)