Wide-Area Crowd Counting: Multi-View Fusion Networks for Counting in Large Scenes

TL;DR

This paper introduces a multi-view neural network framework for crowd counting in large scenes using multiple overlapping cameras, effectively fusing information to produce accurate scene-level density maps.

Contribution

It proposes three fusion models for multi-view crowd counting, including a multi-scale approach and a rotation alignment module, achieving state-of-the-art results.

Findings

State-of-the-art performance on three multi-view datasets.

Effective multi-view fusion improves counting accuracy.

Multi-scale and rotation modules enhance feature alignment.

Abstract

Crowd counting in single-view images has achieved outstanding performance on existing counting datasets. However, single-view counting is not applicable to large and wide scenes (e.g., public parks, long subway platforms, or event spaces) because a single camera cannot capture the whole scene in adequate detail for counting, e.g., when the scene is too large to fit into the field-of-view of the camera, too long so that the resolution is too low on faraway crowds, or when there are too many large objects that occlude large portions of the crowd. Therefore, to solve the wide-area counting task requires multiple cameras with overlapping fields-of-view. In this paper, we propose a deep neural network framework for multi-view crowd counting, which fuses information from multiple camera views to predict a scene-level density map on the ground-plane of the 3D world. We consider three versions of the fusion framework: the late fusion model fuses camera-view density map; the naive early fusion model fuses camera-view feature maps; and the multi-view multi-scale early fusion model ensures that features aligned to the same ground-plane point have consistent scales. A rotation selection module further ensures consistent rotation alignment of the features. We test our 3 fusion models on 3 multi-view counting datasets, PETS2009, DukeMTMC, and a newly collected multi-view counting dataset containing a crowded street intersection. Our methods achieve state-of-the-art results compared to other multi-view counting baselines.