Pro-UIGAN: Progressive Face Hallucination from Occluded Thumbnails

TL;DR

Pro-UIGAN is a multi-stage GAN that progressively super-resolves and inpaints occluded low-resolution faces using facial geometry priors, achieving high-quality results and improving downstream face analysis tasks.

Contribution

It introduces a novel multi-stage framework with a cross-modal transformer for facial priors estimation, enhancing face hallucination from occluded thumbnails.

Findings

Achieves superior visual quality in face super-resolution and inpainting.

Improves accuracy in downstream tasks like face recognition and parsing.

Reduces artifacts and blurriness compared to existing methods.

Abstract

In this paper, we study the task of hallucinating an authentic high-resolution (HR) face from an occluded thumbnail. We propose a multi-stage Progressive Upsampling and Inpainting Generative Adversarial Network, dubbed Pro-UIGAN, which exploits facial geometry priors to replenish and upsample (8*) the occluded and tiny faces (16*16 pixels). Pro-UIGAN iteratively (1) estimates facial geometry priors for low-resolution (LR) faces and (2) acquires non-occluded HR face images under the guidance of the estimated priors. Our multi-stage hallucination network super-resolves and inpaints occluded LR faces in a coarse-to-fine manner, thus reducing unwanted blurriness and artifacts significantly. Specifically, we design a novel cross-modal transformer module for facial priors estimation, in which an input face and its landmark features are formulated as queries and keys, respectively. Such a design encourages joint feature learning across the input facial and landmark features, and deep feature correspondences will be discovered by attention. Thus, facial appearance features and facial geometry priors are learned in a mutual promotion manner. Extensive experiments demonstrate that our Pro-UIGAN achieves visually pleasing HR faces, reaching superior performance in downstream tasks, i.e., face alignment, face parsing, face recognition and expression classification, compared with other state-of-the-art (SotA) methods.