Dual Associated Encoder for Face Restoration

TL;DR

The paper introduces DAEFR, a dual-branch framework for face restoration that effectively leverages both low-quality and high-quality features, outperforming existing methods especially in real-world scenarios.

Contribution

It proposes a novel dual-branch architecture with association training to better bridge the gap between low-quality and high-quality face images.

Findings

DAEFR achieves superior restoration quality on synthetic and real-world datasets.

The auxiliary LQ branch enhances feature extraction from low-quality images.

Association training improves the synergy between branches, boosting performance.

Abstract

Restoring facial details from low-quality (LQ) images has remained a challenging problem due to its ill-posedness induced by various degradations in the wild. The existing codebook prior mitigates the ill-posedness by leveraging an autoencoder and learned codebook of high-quality (HQ) features, achieving remarkable quality. However, existing approaches in this paradigm frequently depend on a single encoder pre-trained on HQ data for restoring HQ images, disregarding the domain gap between LQ and HQ images. As a result, the encoding of LQ inputs may be insufficient, resulting in suboptimal performance. To tackle this problem, we propose a novel dual-branch framework named DAEFR. Our method introduces an auxiliary LQ branch that extracts crucial information from the LQ inputs. Additionally, we incorporate association training to promote effective synergy between the two branches, enhancing code prediction and output quality. We evaluate the effectiveness of DAEFR on both synthetic and real-world datasets, demonstrating its superior performance in restoring facial details. Project page: https://liagm.github.io/DAEFR/