Fully automatic extraction of salient objects from videos in near real-time

TL;DR

This paper introduces a fast, fully automatic video segmentation method that leverages graph cuts, visual saliency, and GPU stream processing to achieve near real-time performance without supervision.

Contribution

The paper presents a novel, efficient automatic video segmentation approach combining graph cuts, saliency-based priors, and GPU implementation for real-time processing.

Findings

Achieves near real-time segmentation speed

Matches the accuracy of semi-automatic methods

Operates without supervision

Abstract

Automatic video segmentation plays an important role in a wide range of computer vision and image processing applications. Recently, various methods have been proposed for this purpose. The problem is that most of these methods are far from real-time processing even for low-resolution videos due to the complex procedures. To this end, we propose a new and quite fast method for automatic video segmentation with the help of 1) efficient optimization of Markov random fields with polynomial time of number of pixels by introducing graph cuts, 2) automatic, computationally efficient but stable derivation of segmentation priors using visual saliency and sequential update mechanism, and 3) an implementation strategy in the principle of stream processing with graphics processor units (GPUs). Test results indicates that our method extracts appropriate regions from videos as precisely as and much faster than previous semi-automatic methods even though any supervisions have not been incorporated.