# BSUV-Net: A Fully-Convolutional Neural Network for Background Subtraction of Unseen Videos

## TL;DR

BSUV-Net is a novel fully-convolutional neural network designed for background subtraction in unseen videos, effectively handling illumination differences and outperforming existing methods on the CDNet-2014 dataset.

## Contribution

This work introduces a supervised background subtraction method that generalizes to unseen videos using a new data-augmentation technique and multi-scale background frames.

## Key findings

- Outperforms state-of-the-art algorithms on unseen videos
- Achieves higher F-measure, recall, and precision on CDNet-2014
- Effectively handles illumination differences with data augmentation

## Abstract

Background subtraction is a basic task in computer vision and video processing often applied as a pre-processing step for object tracking, people recognition, etc. Recently, a number of successful background-subtraction algorithms have been proposed, however nearly all of the top-performing ones are supervised. Crucially, their success relies upon the availability of some annotated frames of the test video during training. Consequently, their performance on completely "unseen" videos is undocumented in the literature. In this work, we propose a new, supervised, background-subtraction algorithm for unseen videos (BSUV-Net) based on a fully-convolutional neural network. The input to our network consists of the current frame and two background frames captured at different time scales along with their semantic segmentation maps. In order to reduce the chance of overfitting, we also introduce a new data-augmentation technique which mitigates the impact of illumination difference between the background frames and the current frame. On the CDNet-2014 dataset, BSUV-Net outperforms state-of-the-art algorithms evaluated on unseen videos in terms of several metrics including F-measure, recall and precision.