A Flow Base Bi-path Network for Cross-scene Video Crowd Understanding in Aerial View

TL;DR

This paper introduces a double-stream bi-path network for cross-scene drone-based crowd understanding, utilizing optical flow, frame difference, data augmentation, and synthetic data to enhance accuracy and generalization in variable environments.

Contribution

The paper proposes a novel bi-path network with optical flow and frame difference branches, combined with data augmentation and synthetic data, to improve crowd counting across diverse drone scenes.

Findings

Achieved a MAE of 12.70 in crowd counting challenge

Synthetic GTAV data effectively improves model performance

Data augmentation enhances model generalization across scenes

Abstract

Drones shooting can be applied in dynamic traffic monitoring, object detecting and tracking, and other vision tasks. The variability of the shooting location adds some intractable challenges to these missions, such as varying scale, unstable exposure, and scene migration. In this paper, we strive to tackle the above challenges and automatically understand the crowd from the visual data collected from drones. First, to alleviate the background noise generated in cross-scene testing, a double-stream crowd counting model is proposed, which extracts optical flow and frame difference information as an additional branch. Besides, to improve the model's generalization ability at different scales and time, we randomly combine a variety of data transformation methods to simulate some unseen environments. To tackle the crowd density estimation problem under extreme dark environments, we introduce synthetic data generated by game Grand Theft Auto V(GTAV). Experiment results show the effectiveness of the virtual data. Our method wins the challenge with a mean absolute error (MAE) of 12.70. Moreover, a comprehensive ablation study is conducted to explore each component's contribution.