Spectral Aware Softmax for Visible-Infrared Person Re-Identification

TL;DR

This paper introduces the spectral-aware softmax loss, enhancing visible-infrared person re-identification by explicitly addressing modality gaps and improving cross-modality feature embedding.

Contribution

It proposes a novel SA-Softmax loss with an asynchronous optimization strategy and modifications to better handle modality discrepancies in VI-ReID.

Findings

SA-Softmax outperforms state-of-the-art methods on RegDB and SYSU-MM01 datasets.

The method effectively reduces modality gap and improves cross-modality matching accuracy.

Experimental results validate the interpretability and robustness of the proposed loss.

Abstract

Visible-infrared person re-identification (VI-ReID) aims to match specific pedestrian images from different modalities. Although suffering an extra modality discrepancy, existing methods still follow the softmax loss training paradigm, which is widely used in single-modality classification tasks. The softmax loss lacks an explicit penalty for the apparent modality gap, which adversely limits the performance upper bound of the VI-ReID task. In this paper, we propose the spectral-aware softmax (SA-Softmax) loss, which can fully explore the embedding space with the modality information and has clear interpretability. Specifically, SA-Softmax loss utilizes an asynchronous optimization strategy based on the modality prototype instead of the synchronous optimization based on the identity prototype in the original softmax loss. To encourage a high overlapping between two modalities, SA-Softmax optimizes each sample by the prototype from another spectrum. Based on the observation and analysis of SA-Softmax, we modify the SA-Softmax with the Feature Mask and Absolute-Similarity Term to alleviate the ambiguous optimization during model training. Extensive experimental evaluations conducted on RegDB and SYSU-MM01 demonstrate the superior performance of the SA-Softmax over the state-of-the-art methods in such a cross-modality condition.