BihoT: A Large-Scale Dataset and Benchmark for Hyperspectral Camouflaged Object Tracking

TL;DR

This paper introduces BihoT, a large-scale hyperspectral camouflaged object tracking dataset, and proposes SPDAN, a spectral prompt-based model that effectively tracks camouflaged objects by focusing on spectral features.

Contribution

The paper presents the first large-scale HCOT dataset BihoT and a novel spectral prompt-based tracker SPDAN, addressing biases in existing datasets and improving tracking of camouflaged objects.

Findings

SPDAN achieves state-of-the-art results on BihoT and other datasets.

BihoT contains 41,912 images across 49 challenging sequences.

SPDAN effectively leverages spectral information for improved tracking.

Abstract

Hyperspectral object tracking (HOT) has exhibited potential in various applications, particularly in scenes where objects are camouflaged. Existing trackers can effectively retrieve objects via band regrouping because of the bias in existing HOT datasets, where most objects tend to have distinguishing visual appearances rather than spectral characteristics. This bias allows the tracker to directly use the visual features obtained from the false-color images generated by hyperspectral images without the need to extract spectral features. To tackle this bias, we find that the tracker should focus on the spectral information when object appearance is unreliable. Thus, we provide a new task called hyperspectral camouflaged object tracking (HCOT) and meticulously construct a large-scale HCOT dataset, termed BihoT, which consists of 41,912 hyperspectral images covering 49 video sequences. The dataset covers various artificial camouflage scenes where objects have similar appearances, diverse spectrums, and frequent occlusion, making it a very challenging dataset for HCOT. Besides, a simple but effective baseline model, named spectral prompt-based distractor-aware network (SPDAN), is proposed, comprising a spectral embedding network (SEN), a spectral prompt-based backbone network (SPBN), and a distractor-aware module (DAM). Specifically, the SEN extracts spectral-spatial features via 3-D and 2-D convolutions. Then, the SPBN fine-tunes powerful RGB trackers with spectral prompts and alleviates the insufficiency of training samples. Moreover, the DAM utilizes a novel statistic to capture the distractor caused by occlusion from objects and background. Extensive experiments demonstrate that our proposed SPDAN achieves state-of-the-art performance on the proposed BihoT and other HOT datasets.