the Shape and Motion of the Lungs and Heart in Dynamic Pulmonary Imaging

and $CO_{2}$ ) between the blood and ambient air; therefore, adequate pulmonary ventilation (air flow) and perfusion (blood flow) are essential for the lungs to function properly. Inadequate pulmonary function may be due to failure in ventilation and perfusion, among other factors. To detect abnormalities of lung ventilation and perfusion, ventilation and perfusion isotope scans are conventionally used, but they can only provide a static, coarse 2D distribution of air and blood in the lungs, and also have a disadvantage of using radioactive isotopes. The primary imaging modality for diagnosing pulmonary disorders is chest X-ray, but the information about pulmonary function (ventilation and perfusion) that may be gleaned from a single chest X-ray is rather limited. To overcome this limitation, this chapter utilizes sequences of digital chest fluoroscopy frames to reveal focal and general pulmonary functional abnormalities by analyzing shape and motion of the lungs and heart.

2 Dynamic Pulmonary Imaging

2.1 Patient Examination

With Dynamic Pulmonary Imaging [1], we can collect a sequence of chest X-ray images of up to $512 \times 512$ pixels at a sampling frequency of 25 Hz with a copper filter of 3 mm. The reason of using a copper filter is to reduce the radiation dose to patients. Two separate examination procedures are used for ventilation and perfusion studies. In the ventilation study, the patient is asked to breathe naturally and normally in a supine position with posteroanterior projection. An image sequence of 55 frames with $192\times 144$ pixels is collected in 4.32 s with a sampling frequency of 12.5 Hz, because in most cases the lungs can complete a full ventilation cycle in 4 s. Based on our experiments, a spatial resolution of $192\times 144$ is sufficient for ventilation analysis. In the perfusion study, the patient is also in a supine position with posteroanterior projection, but with breath held to effectively remove the ventilation effects. An intravenous bolus of X-ray contrast medium may be further used to enhance the perfusion signal strength. Comparing with ventilation, perfusion has a higher frequency, thus requiring a higher temporal sampling rate but a shorter examination time. Furthermore, pulmonary perfusion is asynchronous,¹ demanding a higher spatial resolution. As a result, for perfusion analysis we acquire an image sequence of 52 frames with $384\times 288$ pixels at a sampling frequency of 25 Hz in 2.04 s. The imaging parameters are summarized in Table 1.

Table 1

Dynamic pulmonary imaging parameters used for ventilation and perfusion examinations

Examination	Image size	Temporal sampling frequency (Hz)	Number of frames	Examination time (s)
Ventilation	$\ge \,192\times 144$	12.5	55	4.32
Perfusion	$\ge \,384\times 288$	25	52	2.04

The acquired image sequences may be represented with intensity function $$I(x,y,t)$$

, where $0 \le I \le 255, 1\le x \le \text{ width }$ (192 for ventilation and 384 for perfusion), $1 \le y \le \text{ height }$ (144 for ventilation and 288 for perfusion), and $$t$$

is a discrete time point in $0 \le t \le examtime$ (4.32 s for ventilation and 2.04 s for perfusion). We may also represent it as $$I(x,y,i)$$

, with

the frame index, such that $$t = (i - 1)/f$$

, where

is the sampling frequency of 12.5 Hz for ventilation analysis and 25 Hz for perfusion analysis.

Because of the very short examination time and the use of a copper filter, the radiation dose to the patient is low. The entrance skin dose of a patient is about 0.1 to 0.2 mGy [1]. For comparison, the radiation dose of a normal chest X-ray image varies between 0.1 and 0.2 mGy, and the radiation dose of fluoroscopy is about 2 mGy per minute [1].

2.2 Ventilation and Perfusion Analysis

The 2D image sequence obtained from the patient examination carries valuable information for ventilation and perfusion studies thanks to the physical properties of X-rays: The attenuation of X-rays in air is much lower than in blood and soft tissue. As a result, the average pixel intensity of an area in the lung field varies over time due to the respiratory and cardiac cycles; this variation, called a lung functional signal, reflects the air and blood volume change in the corresponding 2D projectional area of the lung when the patient breathes naturally. When the patient is asked to hold their breath, we observe the perfusion signal disturbed by noise. The ventilation intensity variation depends on the depth of the tidal volume ventilation and on lung area. It is usually between 5 and 15 units in the 8-bit grey scale. The image intensity variation for perfusion is about 3 to 4 units without contrast media. The ventilation signal to noise ratio is about 10:1 and the perfusion signal to noise ratio is about 3:1. This phenomenon is illustrated in Figs. 1, 2 and 3.

Fig. 1

A case in quiet breath. a A region of interest (ROI) in the right lung field and b its corresponding lung functional signal (observation), which reflects the air and blood change in the corresponding lung area over time during the examination due to the physical properties of X-rays. c A set of ventilation parameters can be extracted from the observation b with a ventilation model (see Fig. 4) via optimization, where the observation is indicated with an “o”, the initial guess is plotted as a dashed curve and the final solution as a solid curve. The image gets whiter (higher intensity) during inhalation (more air in the lungs). The ROI shown here is a rectangle, but it may be of arbitrary shape. The ROI may be as large as a whole lung or may be as small as a single pixel

Fig. 2

A case with the breath held and an intravenous bolus of X-ray contrast media. a An ROI (region of interest) in the right lung field and b its corresponding observation—an enhanced lung perfusion signal which (due to the physical properties of X-rays) reflects the blood flow in the corresponding lung area with contrast media. The image gets darker (lower intensity) during the systolic phase (more blood in the lungs). Comparing to ventilation in Fig. 1, the perfusion signal is very noisy and weak (only about 3 intensity-unit variation)

Fig. 3

A case with the breath held but using no X-ray contrast media. a An ROI (region of interest) in the right lung and b its corresponding observation—a perfusion signal reflecting the blood flow in the lung area due to the physical properties of X-rays. It is plotted in the same scale as in Fig. 2 for comparison

Fig. 4

A mathematical function can be used for both ventolation and perfusion analysis. In case of ventilation, it models the volume change of air during lung ventilation (increasing during inhalation and decreasing during exhalation) with five primitive parameters: amplitude A (ventilation strength), downtime D (time for exhalation), uptime U (time for inhalation), timeshift S (time from the starting of examination to the completion of the first inhalation) and level L (mean intensity; its value depending on various factors, medical meaning un-defined). In case of perfusion, it models the blood volume change, increasing during the diastolic phase and decreasing during the systolic phase, but its five free primitive parameters have completely different medical meanings: amplitude A (perfusion strength in the lung area), downtime D (time for the systolic phase in the lung area), uptime U (time for the diastolic phase in the lung area), timeshift S (time from the first image to the completion of the first diastolic phase) and level L (the mean intensity but with no well-defined medical meaning)

We detect ventilation and perfusion abnormalities by extracting meaningful ventilation and perfusion parameters from the lung functional signals. To do so, it is necessary to accurately locate the “turning points” from the signal, but it is challenging due to the existence of both ventilation and perfusion components, in addition to noise. Furthermore, a phase (exhalation, inhalation, diastole, or systole) might not be complete in a signal. For instance, the signal in Fig. 1b does not have a complete exhaling phase. To this end, we introduce a mathematical function (Fig. 4):

$\begin{aligned} M(A, D, U, S, L, t) = \left\{ \begin{array}{ll} A\cos (\pi t'/D) + L &{} \text {if} \;0 \le t' < D \\ A\cos (\pi (t'-D)/U + \pi ) + L &{} \text {if} \;D \le t' < (D + U) \end{array} \right. \end{aligned}$

(1)

where

$\begin{aligned} t' = (t-S)\!\!\!\mod (D + U), \end{aligned}$

(2)

with

for

, and $t' \in [0, D + U)$ , so that ventilation and perfusion parameters can be extracted automatically from lung functional signals via optimization [2–4]. From these five extracted parameters, more parameters can be derived. In case of ventilation, we can compute:

Ventilation Frequency ( $\hat{F_v}$ ) (expressed as the number of breaths per minute):

$\begin{aligned} \hat{F_v} = 60/(\hat{D_v} + \hat{U_v}), \end{aligned}$

(3)
Inhaling Rate ( $\hat{I_v}$ ):

$\begin{aligned} \hat{I_v} = \hat{A_v}/\hat{U_v}, \end{aligned}$

(4)
Exhaling Rate ( $\hat{E_v}$ ):

$\begin{aligned} \hat{E_v} = \hat{A_v}/\hat{D_v}, \end{aligned}$

(5)
Normalized timeshift ( $\hat{H_v}$ ):

$\begin{aligned} \hat{H_v} = \hat{S_v}/(\hat{U_v} + \hat{D_v}), \end{aligned}$

(6)
Updown Ratio ( $\hat{R_v}$ ):

$\begin{aligned} \hat{R_v} = \log _{10}(\hat{U_v}/\hat{D_v}). \end{aligned}$

(7)

We take the logarithm of the “updown ratio” to make it symmetric. All these parameters may be used as ventilation abnormality indicators, but our experiments show that in most cases three parameters, $\hat{A_v}$ (amplitude), $\hat{H_v}$ (normalized timeshift) and $\hat{R_v}$ (updown ratio) are sufficient in revealing ventilation abnormalities.

2.3 Shape and Motion Analysis

An additional, indispensable source of information recorded in the image sequence is the shape and motion of the lungs and heart. This chapter employs this shape and motion information to detect abnormalities of the lungs and heart with the United Snakes technique, which is to be reviewed in Sect. 3. It should be noted that the tasks of lung registration and cardiac motion analysis are challenging, because of the reduced image contrast by the copper filter used in image acquisition to reduce the radiation dose to patients.

3 United Snakes

A snake [5] is a flexible, elastic contour whose behavior is governed by an energy functional, where an internal energy controls the degree of stretchiness and flexibility of the contour while an external energy couples the contour to an image, attracting the snake to features of interest (e.g., intensity edges). The active research in Snakes has resulted in a large family of Snakes algorithms [6–8], including finite element Snakes, B-Snakes, and Fourier Snakes, and related algorithms, such as “live-wire” (also known as “intelligent scissors”) [9–15]. Each of these variants has its strengths and weaknesses.

To extract and model the shape and motion of both the lung and the heart in an accurate and robust manner, the differences between these two organs must be taken into consideration and the most appropriate Snake algorithm selected. For example, the lung boundary is smooth with readily identifiable curved corner regions. Consequently, a Hermite finite-element Snake, which can be constructed directly from user-defined lung boundary points and which can easily control the relative position of its nodal points, is most suitable for lung registration and motion analysis. On the other hand, much of the heart boundary is not visible in the image sequence and there are no readily identifiable landmark points directly on the boundary. In this case, the reduced number of degrees of freedom, high degree of smoothness and control polygon of a B-spline Snake make it an ideal choice. Furthermore, the live-wire algorithm is effective for providing a quick delineation of the lungs and heart and this delineation can then be used by a Snakes algorithm for subsequent segmentation and motion tracking of an image sequence. As a result, a common framework combining the best features of the various Snakes algorithms and the live-wire algorithm is highly desirable.

To this end, United Snakes unifies various Snakes algorithms in a finite element framework, where a particular type of Snake can be derived simply by changing the shape functions at the user level. This unification expands the range of object modeling capabilities within a uniform Snake construction process and provides a uniform Snakes motion tracking mechanism. Consequently, both the shape and motion of the lung and heart can be modeled using a single consistent theoretical and implementational framework. United Snakes is also advantageously combined with live-wire by introducing an effective hard constraint mechanism. The United Snakes framework amplifies the efficiency and reproducibility of the component techniques, and it offers more flexible interactive control while further minimizing user interactions. The reader is referred to [16] for the mathematical details.

In the following sections, along with case studies, we present four applications of United Snakes: lung registration, diaphragm motion analysis, cardiac motion analysis, and cardiac shape analysis.

4 Lung Registration

Through our clinical studies, our expert radiologists have found it convenient and effective to use four rectangles (regions of interests, ROIs) covering the apex, upper, middle and lower lung field in each lung for a quick ventilation examination by inspecting the behaviors of the four corresponding lung functional signals. To facilitate this inspection, we propose an ROI-based analysis with lung registration and division.

4.1 Quick ROI-Based Analysis with Lung Registration and Division

The rapid ROI-based analysis is performed by first interactively delineating the lungs in the first frame with United Snakes (see Fig. 5), and then using the tracking capabilities of Snakes to automatically follow the motion throughout the entire image sequence (Fig. 6). The result is a lung delineation in each frame of the sequence. This step is followed by an automatic division of each lung field into four rectangular regions in each frame (Fig. 7) and an automatic calculation of the average intensity for each region. This process forms four lung functional signals in each lung field, from which ventilation parameters can be automatically extracted as shown in Fig. 8.

4.2 Clinical Case Studies

Fig. 5

Lung delineation with United Snakes. Once the first seed (point 1 in (a)) is placed on the lung boundary, an interactive snake is automatically constructed from the first seed to the current mouse position (position 2 in (a)). With three seeds, the left lung can be delineated (b), and similarly for the right lung (c)

Fig. 6

The tracking result (every tenth image shown). The edge information at the lung apex is rather weak. Furthermore, there is no observable lung motion during quiet breathing. In order to make the snake firmly stick to the apex, it is desirable to maintain a hard constraint point there. Therefore, in the lung delineation as illustrated in Fig. 5, the first seed is usually placed at the lung apex so that it can be utilized as a hard constraint in the tracking process

Fig. 7

Each lung field is divided into four regions with equal heights in each frame so that four lung functional signals in each lung field can be formed. Ventilation parameters are automatically extracted from these signals (see Fig. 8 and Table 2)

Fig. 8

Extracting the ventilation parameters from the eight lung functional signals resulting from the lung division (the observations are indicated with “o”, the initial guesses with dashed curves and the final results with solid curves). The numerical results are given in Table 2

Table 2

Clinical case I: Extracted ventilation parameters