MetaF2N: Blind Image Super-Resolution by Learning Efficient Model Adaptation from Faces

TL;DR

MetaF2N is a novel meta-learning based method that efficiently adapts super-resolution models to real-world low-quality images by leveraging face structures, eliminating the need for extensive face recovery and synthesis steps.

Contribution

The paper introduces MetaF2N, a fast and effective face-guided model adaptation framework for blind image super-resolution that bypasses face recovery and synthesis, requiring only one fine-tuning step.

Findings

MetaF2N outperforms existing methods on synthetic datasets.

MetaF2N achieves superior results on real-world low-quality images.

The approach reduces adaptation time significantly.

Abstract

Due to their highly structured characteristics, faces are easier to recover than natural scenes for blind image super-resolution. Therefore, we can extract the degradation representation of an image from the low-quality and recovered face pairs. Using the degradation representation, realistic low-quality images can then be synthesized to fine-tune the super-resolution model for the real-world low-quality image. However, such a procedure is time-consuming and laborious, and the gaps between recovered faces and the ground-truths further increase the optimization uncertainty. To facilitate efficient model adaptation towards image-specific degradations, we propose a method dubbed MetaF2N, which leverages the contained Faces to fine-tune model parameters for adapting to the whole Natural image in a Meta-learning framework. The degradation extraction and low-quality image synthesis steps are thus circumvented in our MetaF2N, and it requires only one fine-tuning step to get decent performance. Considering the gaps between the recovered faces and ground-truths, we further deploy a MaskNet for adaptively predicting loss weights at different positions to reduce the impact of low-confidence areas. To evaluate our proposed MetaF2N, we have collected a real-world low-quality dataset with one or multiple faces in each image, and our MetaF2N achieves superior performance on both synthetic and real-world datasets. Source code, pre-trained models, and collected datasets are available at https://github.com/yinzhicun/MetaF2N.