Overview

Digital video processing is becoming widely used in many aspects of our nowadays life. The availability of high-computation-power systems allows processing of huge amount of raw data to achieve content based functionalities, such as search and manipulation of objects, semantic description of scenes, detection of unusual events, and recognition of objects. This work applies the probabilistic framework for spatio-temporal video representation to supply infrastructure for face tracking in video sequences. It examines the ability to detect skin color regions in [L,a,b] color space, and handle the inherent over-segmentation problem of the framework.

Figure 1: Over segmentation in the color GMM model

It also compares 3 approaches for using the motion information to extend the feature space to distinguish between blobs that occlude each other and move at different velocity during the clip: Connected Components Analysis, Learning GMM model and Frame-by-Frame tracking. The following figures illustrates the enhanced segmentation capabilities when working in the extended feature space.

Figure 2: Left column represents the labeld pixels drawn with the mean color of the blob in [L,a,b] that they belong to. The middle column is the segmentation result without using motion information, the right column is the segmentation results using motion information.

Figure 3: Applying the extended framework to perform segmentation on the ‘flower garden’ clip. Upper row are the labeled pixels, middle raw is the segmentation results without using motion information, lower raw is the segmentation results using motion information.

For problems or questions regarding this web page contact hayit@eng.tau.ac.il.
Last updated: 01/07/08.