Home
Registration
Super-Resolution
Bacteria Identification
Doppler analysis
MSL segmentation
CGMM
X-ray classification
Cervigrams
Image Clustering
Video Indexing
Object tracking
Gmm&Edge

 

Overview

Visual information, both still-image data as well as video data, require large amounts of memory and computing power for storage and processing. There is thus a need to efficiently index, store, and retrieve the visual information in multimedia databases. This work focuses on video data, video representation and segmentation. Advanced video representation schemes are needed to enable compact video storage as well as a concise model for indexing and retrieval applications. Segmenting an input video stream into interesting “events” is becoming an important research objective. The goal is to progress towards content-based functionalities, such as search and manipulation of objects, semantic description of scenes, detection of unusual events, and recognition of objects.

This work presents a novel statistical framework for modeling and segmenting video content into coherent space-time segments within the video frames and across frames. Unsupervised clustering via Gaussian mixture modeling (GMM), enables the extraction of space-time clusters, or “blobs”, in the representation space, and the extraction of corresponding video-regions in the segmentation of the video content. An important differentiation from existing work is that the video is modeled as a single entity as opposed to a sequence of separate frames. Space and time are treated uniformly in a single-stage modeling framework. The probabilistic space-time video representation scheme is extended further to a piecewise GMM framework in which a succession of GMMs are extracted for the video sequence, instead of a single global model for the entire sequence. The piecewise GMM framework allows for the analysis of extended video sequences and the description of nonlinear, non-convex motion patterns. Using the piecewise GMM framework enables the video to be processed online instead of being processed in batch mode, as in the single global GMM model.

A direct correspondence is shown between the representation space and the image plane, enabling probabilistic video segmentation into representative regions and the segmentation of each individual frame comprising the corresponding frame sequence.

Modeling and segmenting a 100-frame video sequence via the piecewise GMM model. (a) A sample of frames from the original sequence; (b) Modeling the sequence: dynamic blobs at the selected set of frames; (c) Corresponding segmentation maps.

For an overview of the method see the following slides

Publications

bullet

H. Greenspan, J. Goldberger, A. Mayer.
"Probabilistic Space-Time Video Modeling via Piecewise GMM ",
IEEE Transactions on Pattern Analysis and Machine Intelligence,
24(3):384-396, 2004. [pdf]

bullet

H. Greenspan, J. Goldberger, A. Mayer.
"A Probabilistic Framework for Spatio-Temporal Video Representation ", ECCV (4) 2002: 461-475. 

 

horizontal rule

Home Registration Super-Resolution Bacteria Identification Doppler analysis MSL segmentation CGMM X-ray classification Cervigrams Image Clustering Video Indexing Object tracking Gmm&Edge

For problems or questions regarding this web page contact hayit@eng.tau.ac.il.
Last updated: 01/07/08.