Multiple Hypothesis Video Segmentation from Superpixel Flows


Amelio Vazquez-Reina; Shai Avidan; Hanspeter Pfister; Eric Miller


Multiple Hypothesis Video Segmentation (MHVS) is a method for the unsupervised photometric segmentation of video sequences. MHVS segments arbitrarily long video streams by considering only a few frames at a time, and handles the automatic creation, continuation and termination of labels with no user initialization or supervision. The process begins by generating several pre-segmentations per frame and enumerating multiple possible trajectories of pixel regions within a short time window. After assigning each trajectory a score, we let the trajectories compete with each other to segment the sequence. We determine the solution of this segmentation problem as the MAP labeling of a higher-order random fi eld. This framework allows MHVS to achieve spatial and temporal long-range label consistency while operating in an on-line manner. We test MHVS on several videos of natural scenes with arbitrary camera and object motion.

BibTex entry

@proceedings { 266, title = {Multiple Hypothesis Video Segmentation from Superpixel Flows}, year = {2010}, month = {05/09/2010}, publisher = {Springer}, address = {Crete, Greece}, author = {Amelio Vazquez-Reina and Shai Avidan and Hanspeter Pfister and Eric Miller} }