Blind Face Restoration for Under-Display Camera via Dictionary Guided Transformer

TL;DR

This paper introduces a specialized two-stage model and a dictionary-guided transformer for restoring face images captured by under-display cameras, addressing unique degradation challenges and achieving state-of-the-art results.

Contribution

The paper presents a novel UDC degradation model and a dictionary-guided transformer network tailored for blind face restoration in UDC scenarios, with new datasets for training and testing.

Findings

DGFormer achieves state-of-the-art restoration performance.

UDC-DMNet effectively models UDC image degradation.

New datasets facilitate training and evaluation of UDC face restoration methods.

Abstract

By hiding the front-facing camera below the display panel, Under-Display Camera (UDC) provides users with a full-screen experience. However, due to the characteristics of the display, images taken by UDC suffer from significant quality degradation. Methods have been proposed to tackle UDC image restoration and advances have been achieved. There are still no specialized methods and datasets for restoring UDC face images, which may be the most common problem in the UDC scene. To this end, considering color filtering, brightness attenuation, and diffraction in the imaging process of UDC, we propose a two-stage network UDC Degradation Model Network named UDC-DMNet to synthesize UDC images by modeling the processes of UDC imaging. Then we use UDC-DMNet and high-quality face images from FFHQ and CelebA-Test to create UDC face training datasets FFHQ-P/T and testing datasets CelebA-Test-P/T for UDC face restoration. We propose a novel dictionary-guided transformer network named DGFormer. Introducing the facial component dictionary and the characteristics of the UDC image in the restoration makes DGFormer capable of addressing blind face restoration in UDC scenarios. Experiments show that our DGFormer and UDC-DMNet achieve state-of-the-art performance.