Mutual Information Guided Optimal Transport for Unsupervised Visible-Infrared Person Re-identification

TL;DR

This paper introduces a mutual information guided optimal transport method for unsupervised visible-infrared person re-identification, effectively addressing cross-modality variance without annotations through an iterative training and matching strategy.

Contribution

It proposes a novel unsupervised learning framework based on mutual information and optimal transport for cross-modality person re-identification, with a loop training strategy and prototype-based contrastive learning.

Findings

Achieves 60.6% Rank-1 accuracy on SYSU-MM01

Achieves 90.3% Rank-1 accuracy on RegDB

Demonstrates effectiveness without using annotations

Abstract

Unsupervised visible infrared person re-identification (USVI-ReID) is a challenging retrieval task that aims to retrieve cross-modality pedestrian images without using any label information. In this task, the large cross-modality variance makes it difficult to generate reliable cross-modality labels, and the lack of annotations also provides additional difficulties for learning modality-invariant features. In this paper, we first deduce an optimization objective for unsupervised VI-ReID based on the mutual information between the model's cross-modality input and output. With equivalent derivation, three learning principles, i.e., "Sharpness" (entropy minimization), "Fairness" (uniform label distribution), and "Fitness" (reliable cross-modality matching) are obtained. Under their guidance, we design a loop iterative training strategy alternating between model training and cross-modality matching. In the matching stage, a uniform prior guided optimal transport assignment ("Fitness", "Fairness") is proposed to select matched visible and infrared prototypes. In the training stage, we utilize this matching information to introduce prototype-based contrastive learning for minimizing the intra- and cross-modality entropy ("Sharpness"). Extensive experimental results on benchmarks demonstrate the effectiveness of our method, e.g., 60.6% and 90.3% of Rank-1 accuracy on SYSU-MM01 and RegDB without any annotations.