Efficient Motion Prompt Learning for Robust Visual Tracking

TL;DR

This paper introduces a lightweight motion prompt learning method that enhances visual tracking robustness by integrating motion cues with visual features, achieving improved performance with minimal additional training.

Contribution

The paper presents a novel, plug-and-play motion prompt module that can be integrated into existing trackers to leverage temporal motion information effectively.

Findings

Significantly improves tracker robustness across seven benchmarks.

Achieves these improvements with minimal training costs.

Maintains real-time speed with negligible performance sacrifice.

Abstract

Due to the challenges of processing temporal information, most trackers depend solely on visual discriminability and overlook the unique temporal coherence of video data. In this paper, we propose a lightweight and plug-and-play motion prompt tracking method. It can be easily integrated into existing vision-based trackers to build a joint tracking framework leveraging both motion and vision cues, thereby achieving robust tracking through efficient prompt learning. A motion encoder with three different positional encodings is proposed to encode the long-term motion trajectory into the visual embedding space, while a fusion decoder and an adaptive weight mechanism are designed to dynamically fuse visual and motion features. We integrate our motion module into three different trackers with five models in total. Experiments on seven challenging tracking benchmarks demonstrate that the proposed motion module significantly improves the robustness of vision-based trackers, with minimal training costs and negligible speed sacrifice. Code is available at https://github.com/zj5559/Motion-Prompt-Tracking.