AerialMPTNet: Multi-Pedestrian Tracking in Aerial Imagery Using Temporal and Graphical Features

TL;DR

AerialMPTNet is a novel multi-pedestrian tracking method in aerial imagery that combines appearance, movement, and interconnection features, and is validated on a newly introduced large dataset, outperforming existing methods in accuracy and efficiency.

Contribution

The paper introduces AerialMPTNet, a new multi-pedestrian tracking approach that fuses multiple features and provides the largest diverse aerial dataset for benchmarking.

Findings

AerialMPTNet outperforms state-of-the-art methods in accuracy.

AerialMPTNet is more time-efficient than existing algorithms.

The AerialMPT dataset is the largest and most diverse for aerial pedestrian tracking.

Abstract

Multi-pedestrian tracking in aerial imagery has several applications such as large-scale event monitoring, disaster management, search-and-rescue missions, and as input into predictive crowd dynamic models. Due to the challenges such as the large number and the tiny size of the pedestrians (e.g., 4 x 4 pixels) with their similar appearances as well as different scales and atmospheric conditions of the images with their extremely low frame rates (e.g., 2 fps), current state-of-the-art algorithms including the deep learning-based ones are unable to perform well. In this paper, we propose AerialMPTNet, a novel approach for multi-pedestrian tracking in geo-referenced aerial imagery by fusing appearance features from a Siamese Neural Network, movement predictions from a Long Short-Term Memory, and pedestrian interconnections from a GraphCNN. In addition, to address the lack of diverse aerial pedestrian tracking datasets, we introduce the Aerial Multi-Pedestrian Tracking (AerialMPT) dataset consisting of 307 frames and 44,740 pedestrians annotated. We believe that AerialMPT is the largest and most diverse dataset to this date and will be released publicly. We evaluate AerialMPTNet on AerialMPT and KIT AIS, and benchmark with several state-of-the-art tracking methods. Results indicate that AerialMPTNet significantly outperforms other methods on accuracy and time-efficiency.