Towards Authentic Face Restoration with Iterative Diffusion Models and Beyond

TL;DR

This paper introduces IDM, an iterative face restoration system based on diffusion models that enhances realism and high-frequency details, outperforming existing methods in face restoration and benefiting image generation tasks.

Contribution

The paper proposes IDM, a novel iterative diffusion-based face restoration approach that combines intrinsic and extrinsic refinement for more authentic and detailed results.

Findings

Superior performance on blind face restoration tasks.

Improved image generation quality and training stability.

Better results than state-of-the-art methods on FFHQ and ImageNet.

Abstract

An authentic face restoration system is becoming increasingly demanding in many computer vision applications, e.g., image enhancement, video communication, and taking portrait. Most of the advanced face restoration models can recover high-quality faces from low-quality ones but usually fail to faithfully generate realistic and high-frequency details that are favored by users. To achieve authentic restoration, we propose $\textbf{IDM}$, an $\textbf{I}$teratively learned face restoration system based on denoising $\textbf{D}$iffusion $\textbf{M}$odels (DDMs). We define the criterion of an authentic face restoration system, and argue that denoising diffusion models are naturally endowed with this property from two aspects: intrinsic iterative refinement and extrinsic iterative enhancement. Intrinsic learning can preserve the content well and gradually refine the high-quality details, while extrinsic enhancement helps clean the data and improve the restoration task one step further. We demonstrate superior performance on blind face restoration tasks. Beyond restoration, we find the authentically cleaned data by the proposed restoration system is also helpful to image generation tasks in terms of training stabilization and sample quality. Without modifying the models, we achieve better quality than state-of-the-art on FFHQ and ImageNet generation using either GANs or diffusion models.