Escaping Data Scarcity for High-Resolution Heterogeneous Face Hallucination

TL;DR

This paper introduces a novel face hallucination approach for heterogeneous face recognition that leverages facial priors and federated learning to overcome data scarcity and privacy issues, achieving state-of-the-art results.

Contribution

It proposes a new data-efficient face hallucination paradigm utilizing facial priors and federated learning for scalable, privacy-preserving HFR model training.

Findings

Achieves state-of-the-art hallucination results on multiple HFR datasets.

Effectively handles data scarcity and privacy constraints in HFR.

Demonstrates improved recognition performance with the proposed method.

Abstract

In Heterogeneous Face Recognition (HFR), the objective is to match faces across two different domains such as visible and thermal. Large domain discrepancy makes HFR a difficult problem. Recent methods attempting to fill the gap via synthesis have achieved promising results, but their performance is still limited by the scarcity of paired training data. In practice, large-scale heterogeneous face data are often inaccessible due to the high cost of acquisition and annotation process as well as privacy regulations. In this paper, we propose a new face hallucination paradigm for HFR, which not only enables data-efficient synthesis but also allows to scale up model training without breaking any privacy policy. Unlike existing methods that learn face synthesis entirely from scratch, our approach is particularly designed to take advantage of rich and diverse facial priors from visible domain for more faithful hallucination. On the other hand, large-scale training is enabled by introducing a new federated learning scheme to allow institution-wise collaborations while avoiding explicit data sharing. Extensive experiments demonstrate the advantages of our approach in tackling HFR under current data limitations. In a unified framework, our method yields the state-of-the-art hallucination results on multiple HFR datasets.