CrowdTrack: A Benchmark for Difficult Multiple Pedestrian Tracking in Real Scenarios

TL;DR

CrowdTrack is a challenging large-scale dataset of real-world multi-pedestrian tracking scenarios designed to improve algorithm robustness in complex, occluded, and diverse environments, addressing limitations of existing datasets.

Contribution

The paper introduces CrowdTrack, a new large-scale, realistic dataset for multi-pedestrian tracking in complex scenarios, and provides comprehensive analysis and benchmarking of state-of-the-art models.

Findings

Existing models struggle with complex scenarios in CrowdTrack.

Foundation models show varied performance on the dataset.

CrowdTrack enables development of more robust tracking algorithms.

Abstract

Multi-object tracking is a classic field in computer vision. Among them, pedestrian tracking has extremely high application value and has become the most popular research category. Existing methods mainly use motion or appearance information for tracking, which is often difficult in complex scenarios. For the motion information, mutual occlusions between objects often prevent updating of the motion state; for the appearance information, non-robust results are often obtained due to reasons such as only partial visibility of the object or blurred images. Although learning how to perform tracking in these situations from the annotated data is the simplest solution, the existing MOT dataset fails to satisfy this solution. Existing methods mainly have two drawbacks: relatively simple scene composition and non-realistic scenarios. Although some of the video sequences in existing dataset do not have the above-mentioned drawbacks, the number is far from adequate for research purposes. To this end, we propose a difficult large-scale dataset for multi-pedestrian tracking, shot mainly from the first-person view and all from real-life complex scenarios. We name it ``CrowdTrack'' because there are numerous objects in most of the sequences. Our dataset consists of 33 videos, containing a total of 5,185 trajectories. Each object is annotated with a complete bounding box and a unique object ID. The dataset will provide a platform to facilitate the development of algorithms that remain effective in complex situations. We analyzed the dataset comprehensively and tested multiple SOTA models on our dataset. Besides, we analyzed the performance of the foundation models on our dataset. The dataset and project code is released at: https://github.com/loseevaya/CrowdTrack .