CrowdMAC: Masked Crowd Density Completion for Robust Crowd Density Forecasting

TL;DR

CrowdMAC is a novel framework that improves crowd density forecasting by jointly predicting future maps and reconstructing incomplete past maps, using a new masking strategy to handle miss-detections.

Contribution

The paper introduces CrowdMAC, a multi-task learning framework with Temporal-Density-aware Masking for robust crowd density forecasting from incomplete data.

Findings

Achieves state-of-the-art results on seven large-scale datasets.

Demonstrates robustness against synthetic and real miss-detections.

Effective in imputing missing crowd density information.

Abstract

A crowd density forecasting task aims to predict how the crowd density map will change in the future from observed past crowd density maps. However, the past crowd density maps are often incomplete due to the miss-detection of pedestrians, and it is crucial to develop a robust crowd density forecasting model against the miss-detection. This paper presents a MAsked crowd density Completion framework for crowd density forecasting (CrowdMAC), which is simultaneously trained to forecast future crowd density maps from partially masked past crowd density maps (i.e., forecasting maps from past maps with miss-detection) while reconstructing the masked observation maps (i.e., imputing past maps with miss-detection). Additionally, we propose Temporal-Density-aware Masking (TDM), which non-uniformly masks tokens in the observed crowd density map, considering the sparsity of the crowd density maps and the informativeness of the subsequent frames for the forecasting task. Moreover, we introduce multi-task masking to enhance training efficiency. In the experiments, CrowdMAC achieves state-of-the-art performance on seven large-scale datasets, including SDD, ETH-UCY, inD, JRDB, VSCrowd, FDST, and croHD. We also demonstrate the robustness of the proposed method against both synthetic and realistic miss-detections. The code is released at https://fujiry0.github.io/CrowdMAC-project-page.