LiteTrack: Layer Pruning with Asynchronous Feature Extraction for Lightweight and Efficient Visual Tracking

TL;DR

LiteTrack is a lightweight, transformer-based visual tracker that employs asynchronous feature extraction and encoder layer pruning to achieve high accuracy and real-time performance on edge devices.

Contribution

The paper introduces LiteTrack, a novel tracker that combines asynchronous feature extraction with encoder layer pruning for improved efficiency and accuracy.

Findings

LiteTrack-B4 achieves 65.2% AO on GOT-10k at over 100 fps.

LiteTrack-B9 reaches 72.2% AO on GOT-10k and 82.4% AUC on TrackingNet at 171 fps.

The proposed methods outperform existing lightweight trackers in speed and accuracy.

Abstract

The recent advancements in transformer-based visual trackers have led to significant progress, attributed to their strong modeling capabilities. However, as performance improves, running latency correspondingly increases, presenting a challenge for real-time robotics applications, especially on edge devices with computational constraints. In response to this, we introduce LiteTrack, an efficient transformer-based tracking model optimized for high-speed operations across various devices. It achieves a more favorable trade-off between accuracy and efficiency than the other lightweight trackers. The main innovations of LiteTrack encompass: 1) asynchronous feature extraction and interaction between the template and search region for better feature fushion and cutting redundant computation, and 2) pruning encoder layers from a heavy tracker to refine the balnace between performance and speed. As an example, our fastest variant, LiteTrack-B4, achieves 65.2% AO on the GOT-10k benchmark, surpassing all preceding efficient trackers, while running over 100 fps with ONNX on the Jetson Orin NX edge device. Moreover, our LiteTrack-B9 reaches competitive 72.2% AO on GOT-10k and 82.4% AUC on TrackingNet, and operates at 171 fps on an NVIDIA 2080Ti GPU. The code and demo materials will be available at https://github.com/TsingWei/LiteTrack.