Prototype-Driven Multi-Feature Generation for Visible-Infrared Person Re-identification

TL;DR

This paper introduces a Prototype-Driven Multi-feature Generation framework (PDM) for visible-infrared person re-identification, effectively reducing cross-modal discrepancies through diversified feature construction and semantic prototype alignment, achieving state-of-the-art results.

Contribution

The paper proposes a novel PDM framework with Multi-Feature Generation and Prototype Learning modules to improve modality alignment in visible-infrared re-identification.

Findings

Achieves state-of-the-art performance on SYSU-MM01 and LLCM datasets.

Introduces cosine heterogeneity loss to enhance prototype diversity.

Effectively mitigates cross-modal discrepancies in person re-identification.

Abstract

The primary challenges in visible-infrared person re-identification arise from the differences between visible (vis) and infrared (ir) images, including inter-modal and intra-modal variations. These challenges are further complicated by varying viewpoints and irregular movements. Existing methods often rely on horizontal partitioning to align part-level features, which can introduce inaccuracies and have limited effectiveness in reducing modality discrepancies. In this paper, we propose a novel Prototype-Driven Multi-feature generation framework (PDM) aimed at mitigating cross-modal discrepancies by constructing diversified features and mining latent semantically similar features for modal alignment. PDM comprises two key components: Multi-Feature Generation Module (MFGM) and Prototype Learning Module (PLM). The MFGM generates diversity features closely distributed from modality-shared features to represent pedestrians. Additionally, the PLM utilizes learnable prototypes to excavate latent semantic similarities among local features between visible and infrared modalities, thereby facilitating cross-modal instance-level alignment. We introduce the cosine heterogeneity loss to enhance prototype diversity for extracting rich local features. Extensive experiments conducted on the SYSU-MM01 and LLCM datasets demonstrate that our approach achieves state-of-the-art performance. Our codes are available at https://github.com/mmunhappy/ICASSP2025-PDM.