Embedding and Enriching Explicit Semantics for Visible-Infrared Person Re-Identification

TL;DR

This paper introduces a novel framework for visible-infrared person re-identification that leverages explicit semantic embeddings, cross-view semantics compensation, and semantics purification to improve cross-modality matching accuracy.

Contribution

The paper presents a new EEES framework combining large language-vision models, multi-view semantics compensation, and semantics purification for enhanced VIReID performance.

Findings

EEES outperforms existing methods on benchmark datasets.

Explicit semantics embedding improves cross-modality alignment.

Semantics purification reduces noise from conflicting attributes.

Abstract

Visible-infrared person re-identification (VIReID) retrieves pedestrian images with the same identity across different modalities. Existing methods learn visual content solely from images, lacking the capability to sense high-level semantics. In this paper, we propose an Embedding and Enriching Explicit Semantics (EEES) framework to learn semantically rich cross-modality pedestrian representations. Our method offers several contributions. First, with the collaboration of multiple large language-vision models, we develop Explicit Semantics Embedding (ESE), which automatically supplements language descriptions for pedestrians and aligns image-text pairs into a common space, thereby learning visual content associated with explicit semantics. Second, recognizing the complementarity of multi-view information, we present Cross-View Semantics Compensation (CVSC), which constructs multi-view image-text pair representations, establishes their many-to-many matching, and propagates knowledge to single-view representations, thus compensating visual content with its missing cross-view semantics. Third, to eliminate noisy semantics such as conflicting color attributes in different modalities, we design Cross-Modality Semantics Purification (CMSP), which constrains the distance between inter-modality image-text pair representations to be close to that between intra-modality image-text pair representations, further enhancing the modality-invariance of visual content. Finally, experimental results demonstrate the effectiveness and superiority of the proposed EEES.