Visual Style Prompt Learning Using Diffusion Models for Blind Face Restoration

TL;DR

This paper presents a novel blind face restoration method that uses diffusion models to generate visual style prompts, improving the quality of restored facial images by capturing fine details more effectively.

Contribution

The paper introduces a style prompt learning framework with a style-modulated aggregation layer, leveraging diffusion models for enhanced blind face restoration.

Findings

Outperforms existing methods in restoring high-quality facial images.

Effectively captures fine facial details and textures.

Demonstrates superior performance in various restoration scenarios.

Abstract

Blind face restoration aims to recover high-quality facial images from various unidentified sources of degradation, posing significant challenges due to the minimal information retrievable from the degraded images. Prior knowledge-based methods, leveraging geometric priors and facial features, have led to advancements in face restoration but often fall short of capturing fine details. To address this, we introduce a visual style prompt learning framework that utilizes diffusion probabilistic models to explicitly generate visual prompts within the latent space of pre-trained generative models. These prompts are designed to guide the restoration process. To fully utilize the visual prompts and enhance the extraction of informative and rich patterns, we introduce a style-modulated aggregation transformation layer. Extensive experiments and applications demonstrate the superiority of our method in achieving high-quality blind face restoration. The source code is available at \href{https://github.com/LonglongaaaGo/VSPBFR}{https://github.com/LonglongaaaGo/VSPBFR}.