Towards Real-World Blind Face Restoration with Generative Diffusion Prior

TL;DR

This paper introduces BFRffusion, a novel method leveraging pretrained Stable Diffusion for blind face restoration, and presents PFHQ, a privacy-preserving dataset, achieving state-of-the-art results in restoring realistic facial details.

Contribution

The work proposes BFRffusion, a new approach utilizing generative priors from Stable Diffusion, and introduces PFHQ, a balanced, privacy-preserving face dataset for training restoration models.

Findings

BFRffusion outperforms existing methods on synthetic and real datasets.

PFHQ dataset effectively addresses privacy and bias issues.

Code, models, and dataset are publicly available.

Abstract

Blind face restoration is an important task in computer vision and has gained significant attention due to its wide-range applications. Previous works mainly exploit facial priors to restore face images and have demonstrated high-quality results. However, generating faithful facial details remains a challenging problem due to the limited prior knowledge obtained from finite data. In this work, we delve into the potential of leveraging the pretrained Stable Diffusion for blind face restoration. We propose BFRffusion which is thoughtfully designed to effectively extract features from low-quality face images and could restore realistic and faithful facial details with the generative prior of the pretrained Stable Diffusion. In addition, we build a privacy-preserving face dataset called PFHQ with balanced attributes like race, gender, and age. This dataset can serve as a viable alternative for training blind face restoration networks, effectively addressing privacy and bias concerns usually associated with the real face datasets. Through an extensive series of experiments, we demonstrate that our BFRffusion achieves state-of-the-art performance on both synthetic and real-world public testing datasets for blind face restoration and our PFHQ dataset is an available resource for training blind face restoration networks. The codes, pretrained models, and dataset are released at https://github.com/chenxx89/BFRffusion.