Omni Survey for Multimodality Analysis in Visual Object Tracking
Zhangyong Tang, Tianyang Xu, Xuefeng Zhu, Hui Li, Shaochuan Zhao, Tao Zhou, Chunyang Cheng, Xiaojun Wu, Josef Kittler

TL;DR
This paper provides a comprehensive survey of multi-modal visual object tracking (MMVOT), analyzing data modalities, methods, challenges, and benchmarking, and discusses when multi-modal approaches outperform unimodal tracking.
Contribution
It offers the first extensive survey covering all aspects of MMVOT, including dataset analysis, and discusses the conditions under which multi-modal tracking is advantageous.
Findings
Analysis of six MMVOT tasks and 338 references.
Identification of long-tail distribution and lack of animal categories in datasets.
Discussion on when multi-modal tracking outperforms unimodal methods.
Abstract
The development of smart cities has led to the generation of massive amounts of multi-modal data in the context of a range of tasks that enable a comprehensive monitoring of the smart city infrastructure and services. This paper surveys one of the most critical tasks, multi-modal visual object tracking (MMVOT), from the perspective of multimodality analysis. Generally, MMVOT differs from single-modal tracking in four key aspects, data collection, modality alignment and annotation, model designing, and evaluation. Accordingly, we begin with an introduction to the relevant data modalities, laying the groundwork for their integration. This naturally leads to a discussion of challenges of multi-modal data collection, alignment, and annotation. Subsequently, existing MMVOT methods are categorised, based on different ways to deal with visible (RGB) and X modalities: programming the auxiliary…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsVideo Surveillance and Tracking Methods
