M$^5$L: Multi-Modal Multi-Margin Metric Learning for RGBT Tracking

TL;DR

This paper introduces M$^5$L, a novel multi-modal metric learning framework for RGBT tracking that enhances sample distinction and modality fusion, leading to improved tracking accuracy.

Contribution

The paper proposes a multi-margin structured loss and cross-modality constraints to better distinguish confusing samples and fuse RGB and thermal features effectively.

Findings

Outperforms state-of-the-art RGBT trackers on large-scale datasets.

Improves tracking accuracy by enlarging margins between confusing and normal samples.

Enhances feature fusion with modality attention mechanisms.

Abstract

Classifying the confusing samples in the course of RGBT tracking is a quite challenging problem, which hasn't got satisfied solution. Existing methods only focus on enlarging the boundary between positive and negative samples, however, the structured information of samples might be harmed, e.g., confusing positive samples are closer to the anchor than normal positive samples.To handle this problem, we propose a novel Multi-Modal Multi-Margin Metric Learning framework, named M$^5$L for RGBT tracking in this paper. In particular, we design a multi-margin structured loss to distinguish the confusing samples which play a most critical role in tracking performance boosting. To alleviate this problem, we additionally enlarge the boundaries between confusing positive samples and normal ones, between confusing negative samples and normal ones with predefined margins, by exploiting the structured information of all samples in each modality.Moreover, a cross-modality constraint is employed to reduce the difference between modalities and push positive samples closer to the anchor than negative ones from two modalities.In addition, to achieve quality-aware RGB and thermal feature fusion, we introduce the modality attentions and learn them using a feature fusion module in our network. Extensive experiments on large-scale datasets testify that our framework clearly improves the tracking performance and outperforms the state-of-the-art RGBT trackers.