Visual Object Tracking across Diverse Data Modalities: A Review

TL;DR

This survey reviews recent advances in visual object tracking across various data modalities, emphasizing deep learning methods for single-modal and multi-modal scenarios, and provides benchmark comparisons and future insights.

Contribution

It offers a comprehensive categorization and analysis of single-modal and multi-modal VOT frameworks, highlighting recent progress and future directions in the field.

Findings

Benchmark results show varying performance across modalities.

Deep learning methods dominate recent VOT approaches.

Multi-modal VOT is increasingly effective in diverse environments.

Abstract

Visual Object Tracking (VOT) is an attractive and significant research area in computer vision, which aims to recognize and track specific targets in video sequences where the target objects are arbitrary and class-agnostic. The VOT technology could be applied in various scenarios, processing data of diverse modalities such as RGB, thermal infrared and point cloud. Besides, since no one sensor could handle all the dynamic and varying environments, multi-modal VOT is also investigated. This paper presents a comprehensive survey of the recent progress of both single-modal and multi-modal VOT, especially the deep learning methods. Specifically, we first review three types of mainstream single-modal VOT, including RGB, thermal infrared and point cloud tracking. In particular, we conclude four widely-used single-modal frameworks, abstracting their schemas and categorizing the existing inheritors. Then we summarize four kinds of multi-modal VOT, including RGB-Depth, RGB-Thermal, RGB-LiDAR and RGB-Language. Moreover, the comparison results in plenty of VOT benchmarks of the discussed modalities are presented. Finally, we provide recommendations and insightful observations, inspiring the future development of this fast-growing literature.