Under-Display Camera Image Restoration with Scattering Effect

TL;DR

This paper introduces a physically motivated model and a two-branch neural network to effectively restore images captured by under-display cameras affected by display-induced scattering effects, improving image clarity.

Contribution

It models the scattering effect explicitly and designs a dual-branch network to better estimate and suppress scattering in UDC images, advancing restoration techniques.

Findings

Outperforms state-of-the-art UDC restoration methods.

Creates a realistic UDC dataset with ground truth images.

Demonstrates effectiveness on real-world and synthetic data.

Abstract

The under-display camera (UDC) provides consumers with a full-screen visual experience without any obstruction due to notches or punched holes. However, the semi-transparent nature of the display inevitably introduces the severe degradation into UDC images. In this work, we address the UDC image restoration problem with the specific consideration of the scattering effect caused by the display. We explicitly model the scattering effect by treating the display as a piece of homogeneous scattering medium. With the physical model of the scattering effect, we improve the image formation pipeline for the image synthesis to construct a realistic UDC dataset with ground truths. To suppress the scattering effect for the eventual UDC image recovery, a two-branch restoration network is designed. More specifically, the scattering branch leverages global modeling capabilities of the channel-wise self-attention to estimate parameters of the scattering effect from degraded images. While the image branch exploits the local representation advantage of CNN to recover clear scenes, implicitly guided by the scattering branch. Extensive experiments are conducted on both real-world and synthesized data, demonstrating the superiority of the proposed method over the state-of-the-art UDC restoration techniques. The source code and dataset are available at \url{https://github.com/NamecantbeNULL/SRUDC}.