Cross-Spectral Attention for Unsupervised RGB-IR Face Verification and Person Re-identification

TL;DR

This paper introduces an unsupervised cross-spectral framework for RGB-IR face verification and person re-identification, utilizing novel attention, loss, and sparsity techniques to improve discriminative clustering across spectra.

Contribution

It presents a new unsupervised approach combining pseudo triplet loss, cross-spectral attention, and structured sparsity for better cross-spectral biometric matching.

Findings

Achieves superior performance on ARL-VTF and RegDB datasets.

Outperforms some supervised methods in certain cases.

Demonstrates robustness across challenging benchmark datasets.

Abstract

Cross-spectral biometrics, such as matching imagery of faces or persons from visible (RGB) and infrared (IR) bands, have rapidly advanced over the last decade due to increasing sensitivity, size, quality, and ubiquity of IR focal plane arrays and enhanced analytics beyond the visible spectrum. Current techniques for mitigating large spectral disparities between RGB and IR imagery often include learning a discriminative common subspace by exploiting precisely curated data acquired from multiple spectra. Although there are challenges with determining robust architectures for extracting common information, a critical limitation for supervised methods is poor scalability in terms of acquiring labeled data. Therefore, we propose a novel unsupervised cross-spectral framework that combines (1) a new pseudo triplet loss with cross-spectral voting, (2) a new cross-spectral attention network leveraging multiple subspaces, and (3) structured sparsity to perform more discriminative cross-spectral clustering. We extensively compare our proposed RGB-IR biometric learning framework (and its individual components) with recent and previous state-of-the-art models on two challenging benchmark datasets: DEVCOM Army Research Laboratory Visible-Thermal Face Dataset (ARL-VTF) and RegDB person re-identification dataset, and, in some cases, achieve performance superior to completely supervised methods.