Modality-Transition Representation Learning for Visible-Infrared Person Re-Identification

TL;DR

This paper introduces a novel modality-transition representation learning framework for visible-infrared person re-identification, effectively bridging the modality gap without extra parameters and outperforming existing methods.

Contribution

The proposed MTRL framework uses a middle generated image as a modality transmitter and introduces new contrastive and regularization losses for better cross-modal alignment.

Findings

Significantly outperforms existing SOTA methods on three datasets.

Achieves improved accuracy without additional parameters.

Maintains inference speed comparable to the backbone model.

Abstract

Visible-infrared person re-identification (VI-ReID) technique could associate the pedestrian images across visible and infrared modalities in the practical scenarios of background illumination changes. However, a substantial gap inherently exists between these two modalities. Besides, existing methods primarily rely on intermediate representations to align cross-modal features of the same person. The intermediate feature representations are usually create by generating intermediate images (kind of data enhancement), or fusing intermediate features (more parameters, lack of interpretability), and they do not make good use of the intermediate features. Thus, we propose a novel VI-ReID framework via Modality-Transition Representation Learning (MTRL) with a middle generated image as a transmitter from visible to infrared modals, which are fully aligned with the original visible images and similar to the infrared modality. After that, using a modality-transition contrastive loss and a modality-query regularization loss for training, which could align the cross-modal features more effectively. Notably, our proposed framework does not need any additional parameters, which achieves the same inference speed to the backbone while improving its performance on VI-ReID task. Extensive experimental results illustrate that our model significantly and consistently outperforms existing SOTAs on three typical VI-ReID datasets.