In this paper, we present a graph-based image segmentation method (patchcuts) that incorporates features and spatial relations obtained from image patches. Without doubt, graph-based methods have advanced our understanding of image segmentation [3] . The smoothness energy term in most graph-cuts methods is based on pixel intensities only. It is known that pixel intensities can be locally erroneous due to noise and other image acquisition problems. Thus, in these cases, noise may adversely

more »

... the performance of graph-based segmentation methods. In this paper, we propose patch-cuts, a graph-based segmentation method that: i) incorporates image patches in graph cuts; ii) introduces a tunable energy function that consists of intensity, shape, texture, and spatial terms; and iii) integrates a robust class of information theory energy terms. Method: Patch-Cuts Patch Extraction: Patch-cuts incorporates aggregated pixel information, in the form of patches and their properties, into the formalism of graph-based segmentation using the following steps. An image Q(x, y) is split into K non-overlapping patches P = {p k | k = 1, 2, ..., K}. The number of patches is determined by the number of quantized gray scale regions. Patch-cuts computes for each patch p k a set of Haralick [2] gray level co-occurrence matrix (GLCM) features: 1) correlation (ϒ); 2) energy (Γ); 3) contrast (∆); and 4) homogeneity (Ξ). Patch-Based Graph Representation: The proposed method builds a weighted undirected graph. The actual segmentation is obtained by partitioning the graph [1, 3]. Specifically, U L (L k ) determines the energy to assign label L k to the k th patch. The energy function to be minimized consists of U L (L k ) and the sum of the four subsequent patch-based terms The second term U P (L k , L o ) computes the cost of assigning the labels L k , L o to the neighboring patches k and o and N k is the neighborhood system for patch k. Patch-cuts incorporates the Kullback-Leibler and Jensen-Shannon divergence in the energy function. consists of four additive terms. Probability Distributions of Object and Background Regions: Patchcuts uses the negative log-likelihood of the intensity probability distribution as data term: U L (F ) = −ln p(H I k |F ) (foreground) and U L (B) = −ln p(H I k |B) (background). Tunable Energy Function: Patch-cuts takes advantage of extracting region properties which single pixels cannot provide, such as intensity, texture, shape and spatial features. Incorporating region properties in the energy function positions patch-cuts as a highly adjustable graph-cut segmentation algorithm. The four energy terms incur costs based on independent characteristics of a region. We propose a tunable patch-based energy function U P (L k , L o ) which incorporates intensity, texture, shape, and spatial distance information. We incorporate descriptors obtained from all patches in the energy function. The complete energy function E(L) is defined as Figure 1: Sample patch-cuts segmentation results. First column: images with user-supplied fore-and background seed points; second column: images illustrating the patches; third column: binary images showing the patch boundaries; and forth column: the segmentation results using patch-cuts with all weights=1. The number of patches, K is 1,709, 2,929, and 1,220 for the first, second, and third image, respectively. Best if viewed in color. Figure 2: Patch-cuts segmentation results for an axial non-contrast cardiac CT scan, near the aortic root. From the left: seed points, the patch image, the patch boundaries, and the segmentation result with all weights=1. K=11,333. Best if viewed in color. where U L (L k ) is the data term, U I (L k , L o ) captures the intensity information, U S (L k , L o ) denotes the shape energy, U T (L k , L o ) represents the texture energy and U D (L k , L o ) refers to the spatial energy. w I , w T , w S and w D denote the weights for the intensity, texture, shape and distance energy terms, respectively. Each weight is bounded by [0, 1]. Figure 1 shows results of patch-cuts for different images. The weights w I , w T , w S , and w D were set to 1. Figure 2 shows the segmentation of an axial cardiac Computed Tomography (CT) scan acquired near the aortic root. Experiments In this case, all weights=1. Patch-cuts decomposed the scan into 11,333 patches. The fourth image shows the segmentation of the heart. Conclusion We present patch-cuts, a novel graph-based image segmentation method. Patch-cuts introduces a tunable energy function that comprises intensity, shape, texture, and spatial terms. The initial experimental results are encouraging and show that patch-cuts is robust with respect to noise and can be successfully applied to general images as well as to cardiac CT scans. [1] Y.Y. Boykov and M.P. Jolly. Interactive graph cuts for optimal boundary and region segmentation of objects in N-D images. In Proc.