Crowd Counting by Adapting Convolutional Neural Networks with Side Information

TL;DR

This paper introduces an adaptive CNN that incorporates side information like camera angle and height to improve crowd counting accuracy, demonstrating its effectiveness on a new dataset and potential in other vision tasks.

Contribution

The paper proposes an adaptive CNN model that models filter weights as a manifold parametrized by side information, enhancing crowd counting performance.

Findings

ACNN outperforms standard CNN in crowd counting accuracy.

A new dataset with ground-truth camera parameters was collected.

ACNN shows potential in other vision applications like image deconvolution.

Abstract

Computer vision tasks often have side information available that is helpful to solve the task. For example, for crowd counting, the camera perspective (e.g., camera angle and height) gives a clue about the appearance and scale of people in the scene. While side information has been shown to be useful for counting systems using traditional hand-crafted features, it has not been fully utilized in counting systems based on deep learning. In order to incorporate the available side information, we propose an adaptive convolutional neural network (ACNN), where the convolutional filter weights adapt to the current scene context via the side information. In particular, we model the filter weights as a low-dimensional manifold, parametrized by the side information, within the high-dimensional space of filter weights. With the help of side information and adaptive weights, the ACNN can disentangle the variations related to the side information, and extract discriminative features related to the current context. Since existing crowd counting datasets do not contain ground-truth side information, we collect a new dataset with the ground-truth camera angle and height as the side information. On experiments in crowd counting, the ACNN improves counting accuracy compared to a plain CNN with a similar number of parameters. We also apply ACNN to image deconvolution to show its potential effectiveness on other computer vision applications.