Visual information, both still-image data as well as video data, require large amounts of memory and computing power for storage and processing. There is thus a need to efficiently index, store, and retrieve the visual information in multimedia databases. This work focuses on video data, video representation and segmentation. Advanced video representation schemes are needed to enable compact video storage as well as a concise model for indexing and retrieval applications. Segmenting an input video stream into interesting “events” is becoming an important research objective. The goal is to progress towards content-based functionalities, such as search and manipulation of objects, semantic description of scenes, detection of unusual events, and recognition of objects.
This work presents a novel statistical framework for modeling and segmenting video content into coherent space-time segments within the video frames and across frames. Unsupervised clustering via Gaussian mixture modeling (GMM), enables the extraction of space-time clusters, or “blobs”, in the representation space, and the extraction of corresponding video-regions in the segmentation of the video content. An important differentiation from existing work is that the video is modeled as a single entity as opposed to a sequence of separate frames. Space and time are treated uniformly in a single-stage modeling framework. The probabilistic space-time video representation scheme is extended further to a piecewise GMM framework in which a succession of GMMs are extracted for the video sequence, instead of a single global model for the entire sequence. The piecewise GMM framework allows for the analysis of extended video sequences and the description of nonlinear, non-convex motion patterns. Using the piecewise GMM framework enables the video to be processed online instead of being processed in batch mode, as in the single global GMM model.
A direct correspondence is shown between the representation space and the image plane, enabling probabilistic video segmentation into representative regions and the segmentation of each individual frame comprising the corresponding frame sequence.
Modeling and segmenting a 100-frame video sequence via the piecewise GMM model. (a) A sample of frames from the original sequence; (b) Modeling the sequence: dynamic blobs at the selected set of frames; (c) Corresponding segmentation maps.For an overview of the method see the following slides
For problems or
questions regarding this web page contact