Combining Region and Edge Cues for Image Segmentation in a Probabilistic Gaussian Mixture Framework
The purpose of image segmentation is to divide the image into meaningful regions, or objects, by assigning each pixel element to one of these objects. Although image segmentation is important in many applications, it remains one of the most difficult tasks in image processing such that even the most advanced, state-of-the-art, algorithms to date produce only reasonable results. These algorithms are usually based on one of two approaches: Patch-based (similarity based) approach or Boundary-based (dissimilarity based) approach and therefore, enjoys the benefit of one (and suffers from its shortcomings) while not enjoying the benefits of the other. This work focuses on a new segmentation algorithm, which combines patch-based information with edge cues under a probabilistic framework. We use multiple Gaussians for building a statistical model (GMM) with colour (L*a*b*) and location features, and we incorporate edge information based on texture, colour and brightness differences into the EM algorithm. We examine several edge detectors (and their modifications) and edge inclusion methods. We evaluate our results qualitatively and quantitatively on a large dataset of natural images, using the Rand score, and compare our results to the state-of-the-art. We also show some preliminary results for medical data.
Examples of segmentation results (from ‘test’ set): The left column (a) shows the original images. Column (b) shows the segmentation results of the GMM algorithm . Column (c) shows the results of the BSE  and column (d) shows the result of our algorithm, GMM+edge. The number under each image shows its Rand score.
One example of segmentation result and process of a cervigram image. (a) Original image, after detection and elimination of specularities and illumination correction;(b) Original image with expert marking; (c) ‘Blob’ representation overlaid on the modified Canny edge map; (d) Final segmentation map.
Statistical segmentation results using the Rand score over a dataset of 100 natural images. ‘GMM’ is region-based only, ‘BSE’ is the Berkeley Segmentation Engine and ‘GMM+Edge’ is the proposed method
We present an extension to the GMM framework by introducing probabilistic edge information obtained as a preprocessing step, thus presenting a methodology for incorporating edge cues along with region cues within the GMM framework. We believe this is a major step forward in augmenting probabilistic modeling, for image representation and segmentation tasks.We show that the new method outperforms the original one in 80% of cases as well as other existing state-of-the-art algorithms. This is shown quantitatively using the Rand score over a large dataset, and qualitatively by examples.
Our algorithm is general and needs very little adjustments. We expect it to perform equally well or even better for specific applications. The algorithm independently decides the optimal number of Gaussians in the GMM without the need for a predetermined number which may be different for each image. It is entirely probabilistic through the use of the GMM and a probabilistic edge map.
For our direct features we used colour in L*a*b space, and (x,y) location. Through the use of the Berkeley Edge Detector we also incorporated texture and other features indirectly. Incorporating this edge information into the GMM-EM enables the use of multiple Gaussians in the mixture without over-segmentation, through
The proposed method is a major step forward in segmentation methodology, with high potential for significant contribution to specific applications such as automatic segmentation of medical data and the management of medical archives.
For problems or
questions regarding this web page contact