Marginal Space Learning

To localize a 3D object, one needs to estimate nine pose parameters, three for position, three for orientation, and three for anisotropic scaling. We present a simple but elegant method for the efficient estimation of the pose parameters, called Marginal Space Learning (MSL). The MSL partitions the pose estimation problem into a sequence of problems formulated in lower dimension spaces, but of increasingly higher dimensionality, called marginal spaces. Thus, the pose of a 3D object is estimated in three steps: position estimation, position-orientation estimation, and position-orientation-scale estimation. Each search space is efficiently pruned, by preserving after each step only a small number of candidate hypotheses. In this way, the number of testing hypotheses is reduced by six orders of magnitude, compared to the exhaustive full space search.

Steerable Features

Efficient learning based detection methods always rely on efficient image features. Global features, such as Haar wavelet features, are effective to capture the global information (e.g., orientation and scale) of an object. To capture the orientation information of a hypothesis, we should rotate either the volume or the feature templates. It is time consuming to rotate a 3D volume and there is no efficient way to rotate the Haar wavelet feature templates. We introduce the steerable features, which can efficiently capture the position, orientation, and scale information of a pose hypothesis. We sample points from the volume under a sampling pattern and compute at each point local features such as voxel intensity and gradient. The novelty of the steerable features is that they embed the position, orientation, and scale information into the distribution of sampling points, while each individual feature is locally defined. Instead of aligning the volume to a pose hypothesis, we steer the sampling pattern. The steerable features can be extended to incorporate nonrigid deformation parameters or temporal information for the joint spatio-temporal estimation.

Comparison of Marginal Space Learning and Full Space Learning in 2D

The MSL can be extended to detect the bounding box of five degrees of freedom corresponding to 2D objects. We perform a thorough comparison experiment on left ventricle detection in 2D MRI images of the MSL and Full Space Learning (FSL), which exhaustively searches the original full pose parameter space. Experiments show the MSL significantly outperforms the FSL, on both training and test sets. The MSL is about eight times faster than FSL and reduces the box detection error by 78 %. Due to its superior performance, MSL is also often used to detect 2D objects in medical imaging.

Constrained Marginal Space Learning

We present two methods to further constrain the search by exploiting the correlation among object pose parameters, one for the position space and the other for the orientation and scale spaces. The position search space is constrained by the minimum margin calculated from the training set. To constrain the search of the orientation or scale space, we present an example-based strategy by focusing the search to regions with high probability. Using the constrained MSL, we improve the detection speed often by an order of magnitude, resulting in a system that can process one volume in under a second for various 3D object detection tasks.