Unlocking the Potential of Diffusion Priors in Blind Face Restoration

TL;DR

This paper introduces FLIPNET, a unified network that enhances blind face restoration by bridging the gap between diffusion models and real-world degraded images, leading to improved authenticity and fidelity.

Contribution

The work proposes a dual-mode FLIPNET that adapts diffusion priors for more realistic face restoration and degradation synthesis, addressing limitations of prior models.

Findings

Outperforms previous diffusion prior-based BFR methods in authenticity and fidelity.

Better models real-world degradations compared to naive degradation models.

Achieves superior results on benchmark datasets.

Abstract

Although diffusion prior is rising as a powerful solution for blind face restoration (BFR), the inherent gap between the vanilla diffusion model and BFR settings hinders its seamless adaptation. The gap mainly stems from the discrepancy between 1) high-quality (HQ) and low-quality (LQ) images and 2) synthesized and real-world images. The vanilla diffusion model is trained on images with no or less degradations, whereas BFR handles moderately to severely degraded images. Additionally, LQ images used for training are synthesized by a naive degradation model with limited degradation patterns, which fails to simulate complex and unknown degradations in real-world scenarios. In this work, we use a unified network FLIPNET that switches between two modes to resolve specific gaps. In Restoration mode, the model gradually integrates BFR-oriented features and face embeddings from LQ images to achieve authentic and faithful face restoration. In Degradation mode, the model synthesizes real-world like degraded images based on the knowledge learned from real-world degradation datasets. Extensive evaluations on benchmark datasets show that our model 1) outperforms previous diffusion prior based BFR methods in terms of authenticity and fidelity, and 2) outperforms the naive degradation model in modeling the real-world degradations.