RGB-Infrared Cross-Modality Person Re-Identification via Joint Pixel and Feature Alignment

TL;DR

This paper introduces AlignGAN, an end-to-end GAN model that jointly performs pixel and feature alignment to improve RGB-Infrared person re-identification, significantly reducing modality gaps and enhancing identification accuracy.

Contribution

It is the first to jointly model pixel and feature alignment for RGB-IR re-identification using an end-to-end GAN framework, improving over existing methods.

Findings

Achieves 15.4% higher Rank-1 accuracy on SYSU-MM01

Outperforms state-of-the-art methods in mAP

Effectively reduces cross-modality variations

Abstract

RGB-Infrared (IR) person re-identification is an important and challenging task due to large cross-modality variations between RGB and IR images. Most conventional approaches aim to bridge the cross-modality gap with feature alignment by feature representation learning. Different from existing methods, in this paper, we propose a novel and end-to-end Alignment Generative Adversarial Network (AlignGAN) for the RGB-IR RE-ID task. The proposed model enjoys several merits. First, it can exploit pixel alignment and feature alignment jointly. To the best of our knowledge, this is the first work to model the two alignment strategies jointly for the RGB-IR RE-ID problem. Second, the proposed model consists of a pixel generator, a feature generator, and a joint discriminator. By playing a min-max game among the three components, our model is able to not only alleviate the cross-modality and intra-modality variations but also learn identity-consistent features. Extensive experimental results on two standard benchmarks demonstrate that the proposed model performs favorably against state-of-the-art methods. Especially, on SYSU-MM01 dataset, our model can achieve an absolute gain of 15.4% and 12.9% in terms of Rank-1 and mAP.