A Multilayer-Based Framework for Online Background Subtraction with Freely Moving Cameras

TL;DR

This paper introduces a multilayer framework for online background subtraction in videos from moving cameras, enabling multi-object segmentation and outperforming existing methods.

Contribution

It presents a novel multilayer approach that models multiple foreground objects and background simultaneously, using Bayesian filtering and graph-cut techniques.

Findings

Outperforms state-of-the-art methods on challenging sequences

Effectively handles multiple foreground objects in moving camera videos

Provides accurate pixel-wise multi-label segmentation

Abstract

The exponentially increasing use of moving platforms for video capture introduces the urgent need to develop the general background subtraction algorithms with the capability to deal with the moving background. In this paper, we propose a multilayer-based framework for online background subtraction for videos captured by moving cameras. Unlike the previous treatments of the problem, the proposed method is not restricted to binary segmentation of background and foreground, but formulates it as a multi-label segmentation problem by modeling multiple foreground objects in different layers when they appear simultaneously in the scene. We assign an independent processing layer to each foreground object, as well as the background, where both motion and appearance models are estimated, and a probability map is inferred using a Bayesian filtering framework. Finally, Multi-label Graph-cut on Markov Random Field is employed to perform pixel-wise labeling. Extensive evaluation results show that the proposed method outperforms state-of-the-art methods on challenging video sequences.