Real-Time Under-Display Cameras Image Restoration and HDR on Mobile Devices

TL;DR

This paper introduces a lightweight deep learning model for real-time image restoration and HDR processing of under-display camera images on mobile devices, achieving high quality with minimal computational cost.

Contribution

It presents the first efficient, real-time capable deep learning solution for under-display camera image restoration and HDR on smartphones, along with a comprehensive benchmark.

Findings

Model operates in real-time on smartphones

Uses four times fewer operations than existing methods

Achieves competitive results on benchmark datasets

Abstract

The new trend of full-screen devices implies positioning the camera behind the screen to bring a larger display-to-body ratio, enhance eye contact, and provide a notch-free viewing experience on smartphones, TV or tablets. On the other hand, the images captured by under-display cameras (UDCs) are degraded by the screen in front of them. Deep learning methods for image restoration can significantly reduce the degradation of captured images, providing satisfying results for the human eyes. However, most proposed solutions are unreliable or efficient enough to be used in real-time on mobile devices. In this paper, we aim to solve this image restoration problem using efficient deep learning methods capable of processing FHD images in real-time on commercial smartphones while providing high-quality results. We propose a lightweight model for blind UDC Image Restoration and HDR, and we also provide a benchmark comparing the performance and runtime of different methods on smartphones. Our models are competitive on UDC benchmarks while using x4 less operations than others. To the best of our knowledge, we are the first work to approach and analyze this real-world single image restoration problem from the efficiency and production point of view.