A Synthesis-Based Approach for Thermal-to-Visible Face Verification

TL;DR

This paper introduces a robust thermal-to-visible face verification algorithm that outperforms existing methods, studies key factors affecting synthesis, and presents a new extensive multi-spectral face dataset.

Contribution

The paper proposes a novel synthesis-based approach for thermal-to-visible face verification, demonstrating state-of-the-art results and introducing the large MILAB-VTF(B) dataset.

Findings

State-of-the-art performance on ARL-VTF and TUFTS datasets.

Significant outperformance of face frontalization methods in profile-to-frontal verification.

Robustness and wide applicability of the proposed method.

Abstract

In recent years, visible-spectrum face verification systems have been shown to match the performance of experienced forensic examiners. However, such systems are ineffective in low-light and nighttime conditions. Thermal face imagery, which captures body heat emissions, effectively augments the visible spectrum, capturing discriminative facial features in scenes with limited illumination. Due to the increased cost and difficulty of obtaining diverse, paired thermal and visible spectrum datasets, not many algorithms and large-scale benchmarks for low-light recognition are available. This paper presents an algorithm that achieves state-of-the-art performance on both the ARL-VTF and TUFTS multi-spectral face datasets. Importantly, we study the impact of face alignment, pixel-level correspondence, and identity classification with label smoothing for multi-spectral face synthesis and verification. We show that our proposed method is widely applicable, robust, and highly effective. In addition, we show that the proposed method significantly outperforms face frontalization methods on profile-to-frontal verification. Finally, we present MILAB-VTF(B), a challenging multi-spectral face dataset that is composed of paired thermal and visible videos. To the best of our knowledge, with face data from 400 subjects, this dataset represents the most extensive collection of indoor and long-range outdoor thermal-visible face imagery. Lastly, we show that our end-to-end thermal-to-visible face verification system provides strong performance on the MILAB-VTF(B) dataset.