Dynamic Disentangled Fusion Network for RGBT Tracking

TL;DR

The paper introduces DDFNet, a novel RGBT tracking model that dynamically adapts fusion strategies to various challenges by disentangling and optimizing multiple attribute-based fusion models, improving robustness and accuracy.

Contribution

The paper proposes a dynamic disentangled fusion network with attribute-specific fusion models and an adaptive aggregation mechanism for robust RGBT tracking.

Findings

DDFNet outperforms state-of-the-art methods on benchmark datasets.

The attribute-based fusion scheme enhances adaptability to different challenges.

The adaptive aggregation improves fusion effectiveness across scenarios.

Abstract

RGBT tracking usually suffers from various challenging factors of low resolution, similar appearance, extreme illumination, thermal crossover and occlusion, to name a few. Existing works often study complex fusion models to handle challenging scenarios, but can not well adapt to various challenges, which might limit tracking performance. To handle this problem, we propose a novel Dynamic Disentangled Fusion Network called DDFNet, which disentangles the fusion process into several dynamic fusion models via the challenge attributes to adapt to various challenging scenarios, for robust RGBT tracking. In particular, we design six attribute-based fusion models to integrate RGB and thermal features under the six challenging scenarios respectively.Since each fusion model is to deal with the corresponding challenges, such disentangled fusion scheme could increase the fusion capacity without the dependence on large-scale training data. Considering that every challenging scenario also has different levels of difficulty, we propose to optimize the combination of multiple fusion units to form each attribute-based fusion model in a dynamic manner, which could well adapt to the difficulty of the corresponding challenging scenario. To address the issue that which fusion models should be activated in the tracking process, we design an adaptive aggregation fusion module to integrate all features from attribute-based fusion models in an adaptive manner with a three-stage training algorithm. In addition, we design an enhancement fusion module to further strengthen the aggregated feature and modality-specific features. Experimental results on benchmark datasets demonstrate the effectiveness of our DDFNet against other state-of-the-art methods.