Mutimodal Ranking Optimization for Heterogeneous Face Re-identification

TL;DR

This paper introduces a multimodal fusion ranking optimization method for heterogeneous face re-identification, effectively addressing domain discrepancies between NIR and VIS images to improve matching accuracy.

Contribution

It proposes a novel multimodal fusion ranking optimization algorithm with a face translation network and fusion strategies, enhancing re-identification performance across heterogeneous face images.

Findings

Outperforms existing methods on SCface dataset

Effectively utilizes modality complementarity

Improves re-identification accuracy

Abstract

Heterogeneous face re-identification, namely matching heterogeneous faces across disjoint visible light (VIS) and near-infrared (NIR) cameras, has become an important problem in video surveillance application. However, the large domain discrepancy between heterogeneous NIR-VIS faces makes the performance of face re-identification degraded dramatically. To solve this problem, a multimodal fusion ranking optimization algorithm for heterogeneous face re-identification is proposed in this paper. Firstly, we design a heterogeneous face translation network to obtain multimodal face pairs, including NIR-VIS/NIR-NIR/VIS-VIS face pairs, through mutual transformation between NIR-VIS faces. Secondly, we propose linear and non-linear fusion strategies to aggregate initial ranking lists of multimodal face pairs and acquire the optimized re-ranked list based on modal complementarity. The experimental results show that the proposed multimodal fusion ranking optimization algorithm can effectively utilize the complementarity and outperforms some relative methods on the SCface dataset.