Cross-modality Person re-identification with Shared-Specific Feature Transfer

TL;DR

This paper introduces a novel cross-modality person re-identification method that leverages shared and specific features to improve performance, addressing information loss in traditional common-representation approaches.

Contribution

The paper proposes the cm-SSFT algorithm that models and transfers shared and specific features across modalities, enhancing re-identification accuracy.

Findings

Outperforms state-of-the-art methods by 22.5% mAP on SYSU-MM01

Achieves 19.3% mAP improvement on RegDB

Effectively models shared and specific features for cross-modality re-ID

Abstract

Cross-modality person re-identification (cm-ReID) is a challenging but key technology for intelligent video analysis. Existing works mainly focus on learning common representation by embedding different modalities into a same feature space. However, only learning the common characteristics means great information loss, lowering the upper bound of feature distinctiveness. In this paper, we tackle the above limitation by proposing a novel cross-modality shared-specific feature transfer algorithm (termed cm-SSFT) to explore the potential of both the modality-shared information and the modality-specific characteristics to boost the re-identification performance. We model the affinities of different modality samples according to the shared features and then transfer both shared and specific features among and across modalities. We also propose a complementary feature learning strategy including modality adaption, project adversarial learning and reconstruction enhancement to learn discriminative and complementary shared and specific features of each modality, respectively. The entire cm-SSFT algorithm can be trained in an end-to-end manner. We conducted comprehensive experiments to validate the superiority of the overall algorithm and the effectiveness of each component. The proposed algorithm significantly outperforms state-of-the-arts by 22.5% and 19.3% mAP on the two mainstream benchmark datasets SYSU-MM01 and RegDB, respectively.