Towards pedestrian head tracking: A benchmark dataset and a multi-source data fusion network

TL;DR

This paper introduces a large-scale pedestrian head tracking dataset and a novel multi-source data fusion neural network that significantly improves head detection and tracking in crowded scenes.

Contribution

The paper presents the first comprehensive head tracking dataset with diverse scenes and a new CNN-based network that fuses multiple data sources for enhanced accuracy.

Findings

MDFN outperforms existing methods on multiple datasets.

The dataset contains over 50,000 frames with extensive pedestrian head annotations.

Multi-source data fusion significantly improves head detection and tracking performance.

Abstract

Pedestrian detection and tracking in crowded video sequences have many applications, including autonomous driving, robot navigation and pedestrian flow analysis. However, detecting and tracking pedestrians in high-density crowds face many challenges, including intra-class occlusions, complex motions, and diverse poses. Although artificial intelligence (AI) models have achieved great progress in head detection, head tracking datasets and methods are extremely lacking. Existing head datasets have limited coverage of complex pedestrian flows and scenes (e.g., pedestrian interactions, occlusions, and object interference). It is of great importance to develop new head tracking datasets and methods. To address these challenges, we present a Chinese Large-scale Cross-scene Pedestrian Head Tracking dataset (Cchead) and a Multi-source Data Fusion Network (MDFN). The dataset has features that are of considerable interest, including 10 diverse scenes of 50,528 frames with about 2,366,249 heads and 2,358 tracks. Our dataset contains diverse pedestrian moving speeds, directions, and complex crowd pedestrian flows with collision avoidance behaviors. Existing state-of-the-art (SOTA) algorithms are tested and compared on the Cchead dataset. MDFN is the first end-to-end convolutional neural network (CNN)-based head detection and tracking network that jointly trains Red, Green, Blue (RGB) frames, pixel-level motion information, depth maps, and density maps in videos. Ablation experiments confirm the significance of multi-source data fusion. Compared with SOTA pedestrian detection and tracking methods, MDFN achieves superior performance across three datasets: Cchead, Restaurant and Crowd of Heads Dataset (CroHD). To promote further development, we share our source code and trained models for global researchers: https://github.com/kailaisun/Cchead.