DR-BFR: Degradation Representation with Diffusion Models for Blind Face Restoration

TL;DR

This paper introduces DR-BFR, a novel blind face restoration method that leverages degradation representations with diffusion models, significantly improving naturalness and accuracy in restoring high-quality face images from diverse low-quality inputs.

Contribution

The paper proposes a new framework that decouples degradation features from face images using contrastive learning, enhancing diffusion model performance in blind face restoration.

Findings

DR-BFR outperforms existing methods quantitatively and qualitatively.

Degradation representations effectively distinguish various degradation types.

The approach improves the naturalness and accuracy of restored face images.

Abstract

Blind face restoration (BFR) is fundamentally challenged by the extensive range of degradation types and degrees that impact model generalization. Recent advancements in diffusion models have made considerable progress in this field. Nevertheless, a critical limitation is their lack of awareness of specific degradation, leading to potential issues such as unnatural details and inaccurate textures. In this paper, we equip diffusion models with the capability to decouple various degradation as a degradation prompt from low-quality (LQ) face images via unsupervised contrastive learning with reconstruction loss, and demonstrate that this capability significantly improves performance, particularly in terms of the naturalness of the restored images. Our novel restoration scheme, named DR-BFR, guides the denoising of Latent Diffusion Models (LDM) by incorporating Degradation Representation (DR) and content features from LQ images. DR-BFR comprises two modules: 1) Degradation Representation Module (DRM): This module extracts degradation representation with content-irrelevant features from LQ faces and estimates a reasonable distribution in the degradation space through contrastive learning and a specially designed LQ reconstruction. 2) Latent Diffusion Restoration Module (LDRM): This module perceives both degradation features and content features in the latent space, enabling the restoration of high-quality images from LQ inputs. Our experiments demonstrate that the proposed DR-BFR significantly outperforms state-of-the-art methods quantitatively and qualitatively across various datasets. The DR effectively distinguishes between various degradations in blind face inverse problems and provides a reasonably powerful prompt to LDM.