InfoBFR: Real-World Blind Face Restoration via Information Bottleneck

TL;DR

InfoBFR is a versatile plug-and-play framework that significantly improves real-world blind face restoration by addressing neural degradations through information bottleneck and diffusion techniques, outperforming existing methods.

Contribution

The paper introduces InfoBFR, a novel framework that enhances generalization in blind face restoration by effectively mitigating neural degradations using manifold information bottleneck and diffusion LoRA.

Findings

Achieves high-fidelity face restoration without attribute distortions.

Outperforms state-of-the-art GAN and diffusion-based methods.

Operates efficiently with 70ms processing time and 16M parameters.

Abstract

Blind face restoration (BFR) is a highly challenging problem due to the uncertainty of data degradation patterns. Current BFR methods have realized certain restored productions but with inherent neural degradations that limit real-world generalization in complicated scenarios. In this paper, we propose a plug-and-play framework InfoBFR to tackle neural degradations, e.g., prior bias, topological distortion, textural distortion, and artifact residues, which achieves high-generalization face restoration in diverse wild and heterogeneous scenes. Specifically, based on the results from pre-trained BFR models, InfoBFR considers information compression using manifold information bottleneck (MIB) and information compensation with efficient diffusion LoRA to conduct information optimization. InfoBFR effectively synthesizes high-fidelity faces without attribute and identity distortions. Comprehensive experimental results demonstrate the superiority of InfoBFR over state-of-the-art GAN-based and diffusion-based BFR methods, with around 70ms consumption, 16M trainable parameters, and nearly 85% BFR-boosting. It is promising that InfoBFR will be the first plug-and-play restorer universally employed by diverse BFR models to conquer neural degradations.