SFANet: A Spectrum-aware Feature Augmentation Network for Visible-Infrared Person Re-Identification

TL;DR

SFANet introduces a spectrum-aware feature augmentation approach using grayscale-spectrum images and a balanced two-stream network with a novel ranking loss, significantly improving visible-infrared person re-identification accuracy.

Contribution

The paper proposes a novel spectrum-aware feature augmentation network with a balanced two-stream architecture and a bi-directional ranking loss for improved VI-ReID performance.

Findings

Achieves state-of-the-art results on SYSU-MM01 and RegDB datasets.

Reduces modality discrepancy by using grayscale-spectrum images.

Enhances discriminability with BTTR loss and ID embedding methods.

Abstract

Visible-Infrared person re-identification (VI-ReID) is a challenging matching problem due to large modality varitions between visible and infrared images. Existing approaches usually bridge the modality gap with only feature-level constraints, ignoring pixel-level variations. Some methods employ GAN to generate style-consistent images, but it destroys the structure information and incurs a considerable level of noise. In this paper, we explicitly consider these challenges and formulate a novel spectrum-aware feature augementation network named SFANet for cross-modality matching problem. Specifically, we put forward to employ grayscale-spectrum images to fully replace RGB images for feature learning. Learning with the grayscale-spectrum images, our model can apparently reduce modality discrepancy and detect inner structure relations across the different modalities, making it robust to color variations. In feature-level, we improve the conventional two-stream network through balancing the number of specific and sharable convolutional blocks, which preserve the spatial structure information of features. Additionally, a bi-directional tri-constrained top-push ranking loss (BTTR) is embedded in the proposed network to improve the discriminability, which efficiently further boosts the matching accuracy. Meanwhile, we further introduce an effective dual-linear with batch normalization ID embedding method to model the identity-specific information and assits BTTR loss in magnitude stabilizing. On SYSU-MM01 and RegDB datasets, we conducted extensively experiments to demonstrate that our proposed framework contributes indispensably and achieves a very competitive VI-ReID performance.