Divert More Attention to Vision-Language Tracking

TL;DR

This paper demonstrates that pure ConvNets, combined with multimodal vision-language representations, can achieve state-of-the-art tracking performance, challenging the reliance on Transformer-based models and offering a more economical solution.

Contribution

The authors introduce a novel unified-adaptive vision-language representation using ConvNets and a modality mixer, achieving superior tracking performance without Transformers.

Findings

ConvNet-based Siamese tracker improved by 14.5% in SUC on LaSOT

Outperforms several Transformer-based SOTA trackers

Theoretical analysis supports the effectiveness of VL representations

Abstract

Relying on Transformer for complex visual feature learning, object tracking has witnessed the new standard for state-of-the-arts (SOTAs). However, this advancement accompanies by larger training data and longer training period, making tracking increasingly expensive. In this paper, we demonstrate that the Transformer-reliance is not necessary and the pure ConvNets are still competitive and even better yet more economical and friendly in achieving SOTA tracking. Our solution is to unleash the power of multimodal vision-language (VL) tracking, simply using ConvNets. The essence lies in learning novel unified-adaptive VL representations with our modality mixer (ModaMixer) and asymmetrical ConvNet search. We show that our unified-adaptive VL representation, learned purely with the ConvNets, is a simple yet strong alternative to Transformer visual features, by unbelievably improving a CNN-based Siamese tracker by 14.5% in SUC on challenging LaSOT (50.7% > 65.2%), even outperforming several Transformer-based SOTA trackers. Besides empirical results, we theoretically analyze our approach to evidence its effectiveness. By revealing the potential of VL representation, we expect the community to divert more attention to VL tracking and hope to open more possibilities for future tracking beyond Transformer. Code and models will be released at https://github.com/JudasDie/SOTS.