Multi-Attribute guided Thermal Face Image Translation based on Latent Diffusion Model

TL;DR

This paper presents a novel latent diffusion model for thermal to visible face image translation, incorporating multi-attribute classification and a Self-attn Mamba module to enhance image quality, identity preservation, and inference speed in infrared face recognition.

Contribution

Introduces a latent diffusion-based framework with multi-attribute guidance and a new Self-attn Mamba module for improved thermal-to-visible face translation.

Findings

Achieves state-of-the-art results on benchmark datasets.

Enhances image quality and identity preservation.

Speeds up inference with the Self-attn Mamba module.

Abstract

Modern surveillance systems increasingly rely on multi-wavelength sensors and deep neural networks to recognize faces in infrared images captured at night. However, most facial recognition models are trained on visible light datasets, leading to substantial performance degradation on infrared inputs due to significant domain shifts. Early feature-based methods for infrared face recognition proved ineffective, prompting researchers to adopt generative approaches that convert infrared images into visible light images for improved recognition. This paradigm, known as Heterogeneous Face Recognition (HFR), faces challenges such as model and modality discrepancies, leading to distortion and feature loss in generated images. To address these limitations, this paper introduces a novel latent diffusion-based model designed to generate high-quality visible face images from thermal inputs while preserving critical identity features. A multi-attribute classifier is incorporated to extract key facial attributes from visible images, mitigating feature loss during infrared-to-visible image restoration. Additionally, we propose the Self-attn Mamba module, which enhances global modeling of cross-modal features and significantly improves inference speed. Experimental results on two benchmark datasets demonstrate the superiority of our approach, achieving state-of-the-art performance in both image quality and identity preservation.