Towards Robust Blind Face Restoration with Codebook Lookup Transformer
Shangchen Zhou, Kelvin C.K. Chan, Chongyi Li, Chen Change Loy

TL;DR
This paper introduces CodeFormer, a Transformer-based model utilizing a learned codebook prior for robust blind face restoration, effectively handling severe degradation and improving face quality and fidelity.
Contribution
It proposes a novel code prediction paradigm with a codebook prior and a Transformer network for enhanced face restoration, surpassing existing methods in quality and robustness.
Findings
Outperforms state-of-the-art methods in quality and fidelity.
Demonstrates robustness to various severe degradations.
Effective on both synthetic and real-world datasets.
Abstract
Blind face restoration is a highly ill-posed problem that often requires auxiliary guidance to 1) improve the mapping from degraded inputs to desired outputs, or 2) complement high-quality details lost in the inputs. In this paper, we demonstrate that a learned discrete codebook prior in a small proxy space largely reduces the uncertainty and ambiguity of restoration mapping by casting blind face restoration as a code prediction task, while providing rich visual atoms for generating high-quality faces. Under this paradigm, we propose a Transformer-based prediction network, named CodeFormer, to model the global composition and context of the low-quality faces for code prediction, enabling the discovery of natural faces that closely approximate the target faces even when the inputs are severely degraded. To enhance the adaptiveness for different degradation, we also propose a controllable…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsFace recognition and analysis · Facial Nerve Paralysis Treatment and Research · Facial Rejuvenation and Surgery Techniques
