Learning Spatial-Temporal Regularized Tensor Sparse RPCA for Background Subtraction

TL;DR

This paper introduces a novel spatial-temporal regularized tensor sparse RPCA method for background subtraction, enhancing accuracy in dynamic and challenging scenes by enforcing structured sparsity through graph-Laplacian regularizations.

Contribution

It proposes a new tensor RPCA algorithm that incorporates spatial-temporal graph regularizations to better handle dynamic backgrounds and camouflaged objects.

Findings

Outperforms existing methods on six benchmark datasets

Effectively preserves disconnected moving object pixels

Demonstrates robustness in dynamic and jittery scenes

Abstract

Video background subtraction is one of the fundamental problems in computer vision that aims to segment all moving objects. Robust principal component analysis has been identified as a promising unsupervised paradigm for background subtraction tasks in the last decade thanks to its competitive performance in a number of benchmark datasets. Tensor robust principal component analysis variations have improved background subtraction performance further. However, because moving object pixels in the sparse component are treated independently and do not have to adhere to spatial-temporal structured-sparsity constraints, performance is reduced for sequences with dynamic backgrounds, camouflaged, and camera jitter problems. In this work, we present a spatial-temporal regularized tensor sparse RPCA algorithm for precise background subtraction. Within the sparse component, we impose spatial-temporal regularizations in the form of normalized graph-Laplacian matrices. To do this, we build two graphs, one across the input tensor spatial locations and the other across its frontal slices in the time domain. While maximizing the objective function, we compel the tensor sparse component to serve as the spatiotemporal eigenvectors of the graph-Laplacian matrices. The disconnected moving object pixels in the sparse component are preserved by the proposed graph-based regularizations since they both comprise of spatiotemporal subspace-based structure. Additionally, we propose a unique objective function that employs batch and online-based optimization methods to jointly maximize the background-foreground and spatial-temporal regularization components. Experiments are performed on six publicly available background subtraction datasets that demonstrate the superior performance of the proposed algorithm compared to several existing methods. Our source code will be available very soon.