Fully Convolutional Neural Networks for Crowd Segmentation

TL;DR

This paper introduces a fast, fully convolutional neural network for crowd segmentation that processes entire images efficiently and can be extended to other segmentation tasks, combining appearance and motion cues.

Contribution

The paper presents a novel FCNN architecture for crowd segmentation that is fast, scalable to different image sizes, and integrates appearance and motion information in a multi-stage deep learning framework.

Findings

Effective crowd segmentation on large datasets

FCNN achieves lower computation cost than traditional methods

Multi-stage learning improves segmentation accuracy

Abstract

In this paper, we propose a fast fully convolutional neural network (FCNN) for crowd segmentation. By replacing the fully connected layers in CNN with 1 by 1 convolution kernels, FCNN takes whole images as inputs and directly outputs segmentation maps by one pass of forward propagation. It has the property of translation invariance like patch-by-patch scanning but with much lower computation cost. Once FCNN is learned, it can process input images of any sizes without warping them to a standard size. These attractive properties make it extendable to other general image segmentation problems. Based on FCNN, a multi-stage deep learning is proposed to integrate appearance and motion cues for crowd segmentation. Both appearance filters and motion filers are pretrained stage-by-stage and then jointly optimized. Different combination methods are investigated. The effectiveness of our approach and component-wise analysis are evaluated on two crowd segmentation datasets created by us, which include image frames from 235 and 11 scenes, respectively. They are currently the largest crowd segmentation datasets and will be released to the public.