T-FAKE: Synthesizing Thermal Images for Facial Landmarking

TL;DR

This paper introduces T-FAKE, a large-scale synthetic thermal facial image dataset created using a novel RGB2Thermal style transfer method, significantly improving thermal facial landmark detection and analysis.

Contribution

The paper presents a new RGB2Thermal loss function for domain-adaptive thermal image synthesis and the creation of the T-FAKE dataset, enhancing thermal facial landmark detection.

Findings

Improved landmark detection accuracy on thermal images.

High-quality synthetic thermal images with realistic temperature distributions.

Enhanced performance in temperature prediction and perceptual evaluation.

Abstract

Facial analysis is a key component in a wide range of applications such as healthcare, autonomous driving, and entertainment. Despite the availability of various facial RGB datasets, the thermal modality, which plays a crucial role in life sciences, medicine, and biometrics, has been largely overlooked. To address this gap, we introduce the T-FAKE dataset, a new large-scale synthetic thermal dataset with sparse and dense landmarks. To facilitate the creation of the dataset, we propose a novel RGB2Thermal loss function, which enables the domain-adaptive transfer of RGB faces to thermal style. By utilizing the Wasserstein distance between thermal and RGB patches and the statistical analysis of clinical temperature distributions on faces, we ensure that the generated thermal images closely resemble real samples. Using RGB2Thermal style transfer based on our RGB2Thermal loss function, we create the large-scale synthetic thermal T-FAKE dataset with landmark and segmentation annotations. Leveraging our novel T-FAKE dataset, probabilistic landmark prediction, and label adaptation networks, we demonstrate significant improvements in landmark detection methods on thermal images across different landmark conventions. Our models show excellent performance with both sparse 70-point landmarks and dense 478-point landmark annotations. Moreover, our RGB2Thermal loss leads to notable results in terms of perceptual evaluation and temperature prediction.