Global-Local Face Upsampling Network

TL;DR

This paper introduces a deep neural network for face hallucination that effectively reconstructs high-resolution faces from very low-resolution images, especially in challenging uncontrolled conditions, by modeling global and local face constraints end-to-end.

Contribution

The paper presents a novel deep network architecture with separate sub-networks for global face reconstruction and local detail enhancement, optimized with a new adversarial loss function for super-resolution.

Findings

Achieves state-of-the-art results in face hallucination tasks.

Improves visual quality and numerical metrics over previous methods.

Performs well in both controlled and uncontrolled scenarios.

Abstract

Face hallucination, which is the task of generating a high-resolution face image from a low-resolution input image, is a well-studied problem that is useful in widespread application areas. Face hallucination is particularly challenging when the input face resolution is very low (e.g., 10 x 12 pixels) and/or the image is captured in an uncontrolled setting with large pose and illumination variations. In this paper, we revisit the algorithm introduced in [1] and present a deep interpretation of this framework that achieves state-of-the-art under such challenging scenarios. In our deep network architecture the global and local constraints that define a face can be efficiently modeled and learned end-to-end using training data. Conceptually our network design can be partitioned into two sub-networks: the first one implements the holistic face reconstruction according to global constraints, and the second one enhances face-specific details and enforces local patch statistics. We optimize the deep network using a new loss function for super-resolution that combines reconstruction error with a learned face quality measure in adversarial setting, producing improved visual results. We conduct extensive experiments in both controlled and uncontrolled setups and show that our algorithm improves the state of the art both numerically and visually.