Causal graph-based video segmentation

TL;DR

This paper introduces a causal, real-time video segmentation method based on graph-based super-pixel algorithms, ensuring temporal consistency without requiring future frame information.

Contribution

It extends graph-based super-pixel segmentation to a causal, real-time video context, enabling temporally consistent segmentation without future frame access.

Findings

Achieves causal, real-time video segmentation

Ensures temporal consistency in segmentation

Operates without future frame information

Abstract

Numerous approaches in image processing and computer vision are making use of super-pixels as a pre-processing step. Among the different methods producing such over-segmentation of an image, the graph-based approach of Felzenszwalb and Huttenlocher is broadly employed. One of its interesting properties is that the regions are computed in a greedy manner in quasi-linear time. The algorithm may be trivially extended to video segmentation by considering a video as a 3D volume, however, this can not be the case for causal segmentation, when subsequent frames are unknown. We propose an efficient video segmentation approach that computes temporally consistent pixels in a causal manner, filling the need for causal and real time applications.