Deep Multimodal Fusion for Generalizable Person Re-identification

TL;DR

This paper introduces a deep multimodal fusion approach to improve the domain generalization of person re-identification models, enabling better performance in unseen environments by leveraging rich semantic features and a fusion strategy.

Contribution

The paper proposes a novel deep multimodal fusion network with a fusion strategy for better domain generalization in person re-identification, addressing the challenge of deploying models in new, unseen domains.

Findings

Significantly outperforms previous domain generalization methods.

Effective multimodal fusion enhances feature representation and generalization.

Fine-tuning on real-world data further improves performance.

Abstract

Person re-identification plays a significant role in realistic scenarios due to its various applications in public security and video surveillance. Recently, leveraging the supervised or semi-unsupervised learning paradigms, which benefits from the large-scale datasets and strong computing performance, has achieved a competitive performance on a specific target domain. However, when Re-ID models are directly deployed in a new domain without target samples, they always suffer from considerable performance degradation and poor domain generalization. To address this challenge, we propose a Deep Multimodal Fusion network to elaborate rich semantic knowledge for assisting in representation learning during the pre-training. Importantly, a multimodal fusion strategy is introduced to translate the features of different modalities into the common space, which can significantly boost generalization capability of Re-ID model. As for the fine-tuning stage, a realistic dataset is adopted to fine-tune the pre-trained model for better distribution alignment with real-world data. Comprehensive experiments on benchmarks demonstrate that our method can significantly outperform previous domain generalization or meta-learning methods with a clear margin. Our source code will also be publicly available at https://github.com/JeremyXSC/DMF.