Exploring Lightweight Hierarchical Vision Transformers for Efficient Visual Tracking

TL;DR

This paper introduces HiT, a lightweight hierarchical vision transformer for visual tracking that achieves high speed and competitive accuracy on edge devices by using a novel Bridge Module and dual-image position encoding.

Contribution

The paper proposes HiT, a new efficient tracking model with a Bridge Module and dual-image position encoding, enabling high-speed performance on limited hardware.

Findings

Runs at 61 fps on Nvidia Jetson AGX

Achieves 64.6% AUC on LaSOT benchmark

Surpasses previous efficient trackers in accuracy

Abstract

Transformer-based visual trackers have demonstrated significant progress owing to their superior modeling capabilities. However, existing trackers are hampered by low speed, limiting their applicability on devices with limited computational power. To alleviate this problem, we propose HiT, a new family of efficient tracking models that can run at high speed on different devices while retaining high performance. The central idea of HiT is the Bridge Module, which bridges the gap between modern lightweight transformers and the tracking framework. The Bridge Module incorporates the high-level information of deep features into the shallow large-resolution features. In this way, it produces better features for the tracking head. We also propose a novel dual-image position encoding technique that simultaneously encodes the position information of both the search region and template images. The HiT model achieves promising speed with competitive performance. For instance, it runs at 61 frames per second (fps) on the Nvidia Jetson AGX edge device. Furthermore, HiT attains 64.6% AUC on the LaSOT benchmark, surpassing all previous efficient trackers.