LasHeR: A Large-scale High-diversity Benchmark for RGBT Tracking

TL;DR

LasHeR is a large, diverse, and densely annotated benchmark dataset for RGBT tracking, enabling better training and evaluation of tracking algorithms across various conditions and object categories.

Contribution

This work introduces LasHeR, the first large-scale, high-diversity RGBT tracking dataset with extensive annotations, supporting both aligned and unaligned tracking research.

Findings

Evaluated 12 RGBT tracking algorithms on LasHeR

Identified challenges and research directions in RGBT tracking

Provided a new benchmark for alignment-free RGBT tracking

Abstract

RGBT tracking receives a surge of interest in the computer vision community, but this research field lacks a large-scale and high-diversity benchmark dataset, which is essential for both the training of deep RGBT trackers and the comprehensive evaluation of RGBT tracking methods. To this end, we present a Large-scale High-diversity benchmark for RGBT tracking (LasHeR) in this work. LasHeR consists of 1224 visible and thermal infrared video pairs with more than 730K frame pairs in total. Each frame pair is spatially aligned and manually annotated with a bounding box, making the dataset well and densely annotated. LasHeR is highly diverse capturing from a broad range of object categories, camera viewpoints, scene complexities and environmental factors across seasons, weathers, day and night. We conduct a comprehensive performance evaluation of 12 RGBT tracking algorithms on the LasHeR dataset and present detailed analysis to clarify the research room in RGBT tracking. In addition, we release the unaligned version of LasHeR to attract the research interest for alignment-free RGBT tracking, which is a more practical task in real-world applications. The datasets and evaluation protocols are available at: https://github.com/BUGPLEASEOUT/LasHeR.