Robust Foreground-Background Separation for Severely-Degraded Videos Using Convolutional Sparse Representation Modeling

TL;DR

This paper introduces a robust foreground-background separation method for severely degraded videos using convolutional sparse representation, effectively handling low frame rates and various noise types.

Contribution

It proposes a novel CSR-based foreground model and an optimization framework that captures both specific and general features, improving separation accuracy in noisy, low-frame-rate videos.

Findings

Outperforms existing methods on infrared videos

Effective in separating foreground and background under noise

Handles low frame rate and multiple noise types

Abstract

This paper proposes a foreground-background separation (FBS) method with a novel foreground model based on convolutional sparse representation (CSR). In order to analyze the dynamic and static components of videos acquired under undesirable conditions, such as hardware, environmental, and power limitations, it is essential to establish an FBS method that can handle videos with low frame rates and various types of noise. Existing FBS methods have two limitations that prevent us from accurately separating foreground and background components from such degraded videos. First, they only capture either data-specific or general features of the components. Second, they do not include explicit models for various types of noise to remove them in the FBS process. To this end, we propose a robust FBS method with a CSR-based foreground model. This model can adaptively capture specific spatial structures scattered in imaging data. Then, we formulate FBS as a constrained multiconvex optimization problem that incorporates CSR, functions that capture general features, and explicit noise characterization functions for multiple types of noise. Thanks to these functions, our method captures both data-specific and general features to accurately separate the components from various types of noise even under low frame rates. To obtain a solution of the optimization problem, we develop an algorithm that alternately solves its two convex subproblems by newly established algorithms. Experiments demonstrate the superiority of our method over existing methods using two types of degraded videos: infrared and microscope videos.