Characteristic Regularisation for Super-Resolving Face Images

TL;DR

This paper introduces Characteristic Regularisation, a novel approach that improves face image super-resolution on genuine low-resolution data by effectively separating and optimizing characteristic consistency and super-resolution tasks.

Contribution

The paper proposes a new method that separates characteristic regularisation from super-resolution, enhancing training effectiveness and performance on real-world low-resolution face images.

Findings

Outperforms state-of-the-art SR and UDA models on genuine LR face data

Effective separation of tasks improves training stability and results

Demonstrates superior performance on both genuine and artificial LR images

Abstract

Existing facial image super-resolution (SR) methods focus mostly on improving artificially down-sampled low-resolution (LR) imagery. Such SR models, although strong at handling artificial LR images, often suffer from significant performance drop on genuine LR test data. Previous unsupervised domain adaptation (UDA) methods address this issue by training a model using unpaired genuine LR and HR data as well as cycle consistency loss formulation. However, this renders the model overstretched with two tasks: consistifying the visual characteristics and enhancing the image resolution. Importantly, this makes the end-to-end model training ineffective due to the difficulty of back-propagating gradients through two concatenated CNNs. To solve this problem, we formulate a method that joins the advantages of conventional SR and UDA models. Specifically, we separate and control the optimisations for characteristics consistifying and image super-resolving by introducing Characteristic Regularisation (CR) between them. This task split makes the model training more effective and computationally tractable. Extensive evaluations demonstrate the performance superiority of our method over state-of-the-art SR and UDA models on both genuine and artificial LR facial imagery data.