Deep Learning for Visual Tracking: A Comprehensive Survey

TL;DR

This comprehensive survey reviews deep learning-based visual tracking methods, benchmark datasets, and evaluation metrics, analyzing their characteristics, performance, and challenges to guide future research and practical applications.

Contribution

It systematically investigates DL-based visual tracking techniques, benchmarks, and evaluations, providing a detailed analysis and comparison to inform future research directions.

Findings

Deep learning methods have significantly advanced visual tracking performance.

Benchmark datasets and evaluation metrics are crucial for assessing tracking methods.

Analysis reveals strengths and weaknesses of state-of-the-art trackers under various scenarios.

Abstract

Visual target tracking is one of the most sought-after yet challenging research topics in computer vision. Given the ill-posed nature of the problem and its popularity in a broad range of real-world scenarios, a number of large-scale benchmark datasets have been established, on which considerable methods have been developed and demonstrated with significant progress in recent years -- predominantly by recent deep learning (DL)-based methods. This survey aims to systematically investigate the current DL-based visual tracking methods, benchmark datasets, and evaluation metrics. It also extensively evaluates and analyzes the leading visual tracking methods. First, the fundamental characteristics, primary motivations, and contributions of DL-based methods are summarized from nine key aspects of: network architecture, network exploitation, network training for visual tracking, network objective, network output, exploitation of correlation filter advantages, aerial-view tracking, long-term tracking, and online tracking. Second, popular visual tracking benchmarks and their respective properties are compared, and their evaluation metrics are summarized. Third, the state-of-the-art DL-based methods are comprehensively examined on a set of well-established benchmarks of OTB2013, OTB2015, VOT2018, LaSOT, UAV123, UAVDT, and VisDrone2019. Finally, by conducting critical analyses of these state-of-the-art trackers quantitatively and qualitatively, their pros and cons under various common scenarios are investigated. It may serve as a gentle use guide for practitioners to weigh when and under what conditions to choose which method(s). It also facilitates a discussion on ongoing issues and sheds light on promising research directions.