MSO: Multi-Feature Space Joint Optimization Network for RGB-Infrared Person Re-Identification

TL;DR

This paper introduces a novel multi-feature space joint optimization network for RGB-infrared person re-identification, enhancing feature learning in both single-modality and common spaces to improve cross-modality recognition accuracy.

Contribution

The paper proposes the first explicit optimization in single-modality feature space and introduces a cross-modality contrastive-center loss for improved cross-modality discrimination.

Findings

Significantly outperforms state-of-the-art on SYSU-MM01 dataset.

Achieves superior results on RegDB dataset.

Effective enhancement of modality-sharable features in shallow layers.

Abstract

The RGB-infrared cross-modality person re-identification (ReID) task aims to recognize the images of the same identity between the visible modality and the infrared modality. Existing methods mainly use a two-stream architecture to eliminate the discrepancy between the two modalities in the final common feature space, which ignore the single space of each modality in the shallow layers. To solve it, in this paper, we present a novel multi-feature space joint optimization (MSO) network, which can learn modality-sharable features in both the single-modality space and the common space. Firstly, based on the observation that edge information is modality-invariant, we propose an edge features enhancement module to enhance the modality-sharable features in each single-modality space. Specifically, we design a perceptual edge features (PEF) loss after the edge fusion strategy analysis. According to our knowledge, this is the first work that proposes explicit optimization in the single-modality feature space on cross-modality ReID task. Moreover, to increase the difference between cross-modality distance and class distance, we introduce a novel cross-modality contrastive-center (CMCC) loss into the modality-joint constraints in the common feature space. The PEF loss and CMCC loss jointly optimize the model in an end-to-end manner, which markedly improves the network's performance. Extensive experiments demonstrate that the proposed model significantly outperforms state-of-the-art methods on both the SYSU-MM01 and RegDB datasets.