GLAD: Generative Language-Assisted Visual Tracking for Low-Semantic Templates

TL;DR

GLAD introduces a generative language-assisted visual tracking method using diffusion models to improve the fusion of visual and linguistic information, especially for low-semantic templates, achieving state-of-the-art results.

Contribution

This paper presents a novel diffusion-based generative fusion approach for vision-language tracking, addressing limitations of existing methods with low-semantic images.

Findings

Achieves state-of-the-art performance on multiple benchmarks.

Effectively restores blurry and ambiguous template images.

Maintains impressive inference speed.

Abstract

Vision-language tracking has gained increasing attention in many scenarios. This task simultaneously deals with visual and linguistic information to localize objects in videos. Despite its growing utility, the development of vision-language tracking methods remains in its early stage. Current vision-language trackers usually employ Transformer architectures for interactive integration of template, search, and text features. However, persistent challenges about low-semantic images including prevalent image blurriness, low resolution and so on, may compromise model performance through degraded cross-modal understanding. To solve this problem, language assistance is usually used to deal with the obstacles posed by low-semantic images. However, due to the existing gap between current textual and visual features, direct concatenation and fusion of these features may have limited effectiveness. To address these challenges, we introduce a pioneering Generative Language-AssisteD tracking model, GLAD, which utilizes diffusion models for the generative multi-modal fusion of text description and template image to bolster compatibility between language and image and enhance template image semantic information. Our approach demonstrates notable improvements over the existing fusion paradigms. Blurry and semantically ambiguous template images can be restored to improve multi-modal features in the generative fusion paradigm. Experiments show that our method establishes a new state-of-the-art on multiple benchmarks and achieves an impressive inference speed. The code and models will be released at: https://github.com/Confetti-lxy/GLAD