Face Super-Resolution Guided by 3D Facial Priors

TL;DR

This paper introduces a face super-resolution method that leverages 3D facial priors to better capture facial structures and identity, especially under large pose variations, resulting in improved performance.

Contribution

It is the first to incorporate 3D morphable face priors into super-resolution, enhancing accuracy and convergence speed across various network architectures.

Findings

Outperforms state-of-the-art methods in face super-resolution

Effectively handles large pose variations in facial images

Improves convergence speed of super-resolution networks

Abstract

State-of-the-art face super-resolution methods employ deep convolutional neural networks to learn a mapping between low- and high- resolution facial patterns by exploring local appearance knowledge. However, most of these methods do not well exploit facial structures and identity information, and struggle to deal with facial images that exhibit large pose variations. In this paper, we propose a novel face super-resolution method that explicitly incorporates 3D facial priors which grasp the sharp facial structures. Our work is the first to explore 3D morphable knowledge based on the fusion of parametric descriptions of face attributes (e.g., identity, facial expression, texture, illumination, and face pose). Furthermore, the priors can easily be incorporated into any network and are extremely efficient in improving the performance and accelerating the convergence speed. Firstly, a 3D face rendering branch is set up to obtain 3D priors of salient facial structures and identity knowledge. Secondly, the Spatial Attention Module is used to better exploit this hierarchical information (i.e., intensity similarity, 3D facial structure, and identity content) for the super-resolution problem. Extensive experiments demonstrate that the proposed 3D priors achieve superior face super-resolution results over the state-of-the-arts.