Foreground Segmentation Using a Triplet Convolutional Neural Network for Multiscale Feature Encoding

TL;DR

This paper introduces a triplet convolutional neural network with multiscale feature encoding for robust foreground segmentation, outperforming state-of-the-art methods in challenging scenarios like illumination changes and camouflage.

Contribution

It proposes a novel encoder-decoder neural network architecture using a pre-trained VGG-16 in a triplet framework for improved foreground segmentation.

Findings

Achieved an average F-Measure of 0.9770 on the Change Detection 2014 Challenge.

Outperformed all existing state-of-the-art methods in the challenge.

The method works effectively with limited training samples.

Abstract

A common approach for moving objects segmentation in a scene is to perform a background subtraction. Several methods have been proposed in this domain. However, they lack the ability of handling various difficult scenarios such as illumination changes, background or camera motion, camouflage effect, shadow etc. To address these issues, we propose a robust and flexible encoder-decoder type neural network based approach. We adapt a pre-trained convolutional network, i.e. VGG-16 Net, under a triplet framework in the encoder part to embed an image in multiple scales into the feature space and use a transposed convolutional network in the decoder part to learn a mapping from feature space to image space. We train this network end-to-end by using only a few training samples. Our network takes an RGB image in three different scales and produces a foreground segmentation probability mask for the corresponding image. In order to evaluate our model, we entered the Change Detection 2014 Challenge (changedetection.net) and our method outperformed all the existing state-of-the-art methods by an average F-Measure of 0.9770. Our source code will be made publicly available at https://github.com/lim-anggun/FgSegNet.