HHTrack: Hyperspectral Object Tracking Using Hybrid Attention

TL;DR

This paper introduces HHTrack, a hyperspectral object tracking method that uses hybrid attention and spectral-spatial fusion to improve tracking accuracy by leveraging rich spectral information.

Contribution

The paper proposes a novel hyperspectral hybrid attention module and a spectral bands fusion module, advancing the integration of transformers and hyperspectral data for robust object tracking.

Findings

Achieves state-of-the-art results on multiple hyperspectral tracking datasets.

Demonstrates the effectiveness of hybrid attention in hyperspectral feature extraction.

Shows improved robustness over existing methods in various spectral conditions.

Abstract

Hyperspectral imagery provides abundant spectral information beyond the visible RGB bands, offering rich discriminative details about objects in a scene. Leveraging such data has the potential to enhance visual tracking performance. In this paper, we propose a hyperspectral object tracker based on hybrid attention (HHTrack). The core of HHTrack is a hyperspectral hybrid attention (HHA) module that unifies feature extraction and fusion within one component through token interactions. A hyperspectral bands fusion (HBF) module is also introduced to selectively aggregate spatial and spectral signatures from the full hyperspectral input. Extensive experiments demonstrate the state-of-the-art performance of HHTrack on benchmark Near Infrared (NIR), Red Near Infrared (Red-NIR), and Visible (VIS) hyperspectral tracking datasets. Our work provides new insights into harnessing the strengths of transformers and hyperspectral fusion to advance robust object tracking.