Joint Face Super-Resolution and Deblurring Using a Generative Adversarial Network

TL;DR

This paper introduces a GAN-based method for joint face super-resolution and deblurring, producing realistic high-resolution facial images with diverse variations, surpassing traditional PSNR-focused approaches.

Contribution

It presents a novel adversarial framework that simultaneously enhances resolution and reduces blur in facial images, incorporating dual decoders and discriminators for improved realism.

Findings

Generates realistic high-resolution facial images.

Produces diverse facial image variations.

Outperforms PSNR-focused methods in realism.

Abstract

Facial image super-resolution (SR) is an important preprocessing for facial image analysis, face recognition, and image-based 3D face reconstruction. Recent convolutional neural network (CNN) based method has shown excellent performance by learning mapping relation using pairs of low-resolution (LR) and high-resolution (HR) facial images. However, since the HR facial image reconstruction using CNN is conventionally aimed to increase the PSNR and SSIM metrics, the reconstructed HR image might not be realistic even with high scores. An adversarial framework is proposed in this study to reconstruct the HR facial image by simultaneously generating an HR image with and without blur. First, the spatial resolution of the LR facial image is increased by eight times using a five-layer CNN. Then, the encoder extracts the features of the up-scaled image. These features are finally sent to two branches (decoders) to generate an HR facial image with and without blur. In addition, local and global discriminators are combined to focus on the reconstruction of HR facial structures. Experiment results show that the proposed algorithm generates a realistic HR facial image. Furthermore, the proposed method can generate a variety of different facial images.