Video Object Segmentation using Supervoxel-Based Gerrymandering

TL;DR

This paper introduces supervoxel gerrymandering, a novel unsupervised video object segmentation method that leverages supervoxels for effective space-time information aggregation, outperforming existing methods.

Contribution

It thoroughly analyzes supervoxel use in video segmentation and proposes a new approach that surpasses current unsupervised and many supervised techniques.

Findings

Outperforms all known unsupervised methods on DAVIS dataset

Effective use of supervoxels for space-time information aggregation

Provides insights into supervoxel scale and consensus mechanisms

Abstract

Pixels operate locally. Superpixels have some potential to collect information across many pixels; supervoxels have more potential by implicitly operating across time. In this paper, we explore this well established notion thoroughly analyzing how supervoxels can be used in place of and in conjunction with other means of aggregating information across space-time. Focusing on the problem of strictly unsupervised video object segmentation, we devise a method called supervoxel gerrymandering that links masks of foregroundness and backgroundness via local and non-local consensus measures. We pose and answer a series of critical questions about the ability of supervoxels to adequately sway local voting; the questions regard type and scale of supervoxels as well as local versus non-local consensus, and the questions are posed in a general way so as to impact the broader knowledge of the use of supervoxels in video understanding. We work with the DAVIS dataset and find that our analysis yields an unsupervised method that outperforms all other known unsupervised methods and even many supervised ones.