Shape-centered Representation Learning for Visible-Infrared Person Re-identification

TL;DR

This paper introduces a novel framework called Shape-centered Representation Learning (ScRL) for visible-infrared person re-identification, effectively integrating shape and appearance features to improve robustness and accuracy across multiple datasets.

Contribution

The paper proposes the ScRL framework with Infrared Shape Restoration, Shape Feature Propagation, and Appearance Feature Enhancement to better combine shape and appearance features in VI-ReID.

Findings

Achieves state-of-the-art Rank-1 and mAP on SYSU-MM01 dataset.

Demonstrates significant performance improvements on HITSZ-VCM and RegDB datasets.

Effectively integrates shape features to enhance robustness against modality variations.

Abstract

Visible-Infrared Person Re-Identification (VI-ReID) plays a critical role in all-day surveillance systems. However, existing methods primarily focus on learning appearance features while overlooking body shape features, which not only complement appearance features but also exhibit inherent robustness to modality variations. Despite their potential, effectively integrating shape and appearance features remains challenging. Appearance features are highly susceptible to modality variations and background noise, while shape features often suffer from inaccurate infrared shape estimation due to the limitations of auxiliary models. To address these challenges, we propose the Shape-centered Representation Learning (ScRL) framework, which enhances VI-ReID performance by innovatively integrating shape and appearance features. Specifically, we introduce Infrared Shape Restoration (ISR) to restore inaccuracies in infrared body shape representations at the feature level by leveraging infrared appearance features. In addition, we propose Shape Feature Propagation (SFP), which enables the direct extraction of shape features from original images during inference with minimal computational complexity. Furthermore, we design Appearance Feature Enhancement (AFE), which utilizes shape features to emphasize shape-related appearance features while effectively suppressing identity-unrelated noise. Benefiting from the effective integration of shape and appearance features, ScRL demonstrates superior performance through extensive experiments. On the SYSU-MM01, HITSZ-VCM, and RegDB datasets, it achieves Rank-1 (mAP) accuracies of 76.1% (72.6%), 71.2% (52.9%), and 92.4% (86.7%), respectively, surpassing existing state-of-the-art methods.