Transform Domain Pyramidal Dilated Convolution Networks For Restoration of Under Display Camera Images

TL;DR

This paper introduces two novel deep learning networks employing pyramidal dilated convolutions for restoring images captured by under-display cameras, significantly improving image quality for different display technologies.

Contribution

It proposes two specialized neural network architectures tailored for P-OLED and T-OLED under-display camera image restoration, achieving top challenge rankings.

Findings

First method achieved top performance in ECCV 2020 challenge.

Second method ranked fourth in the ECCV challenge.

Both methods significantly improved image restoration quality.

Abstract

Under-display camera (UDC) is a novel technology that can make digital imaging experience in handheld devices seamless by providing large screen-to-body ratio. UDC images are severely degraded owing to their positioning under a display screen. This work addresses the restoration of images degraded as a result of UDC imaging. Two different networks are proposed for the restoration of images taken with two types of UDC technologies. The first method uses a pyramidal dilated convolution within a wavelet decomposed convolutional neural network for pentile-organic LED (P-OLED) based display system. The second method employs pyramidal dilated convolution within a discrete cosine transform based dual domain network to restore images taken using a transparent-organic LED (T-OLED) based UDC system. The first method produced very good quality restored images and was the winning entry in European Conference on Computer Vision (ECCV) 2020 challenge on image restoration for Under-display Camera - Track 2 - P-OLED evaluated based on PSNR and SSIM. The second method scored fourth position in Track-1 (T-OLED) of the challenge evaluated based on the same metrics.