UCMNet: Uncertainty-Aware Context Memory Network for Under-Display Camera Image Restoration

TL;DR

UCMNet is a lightweight neural network that uses uncertainty estimation to adaptively restore high-frequency details in images captured by under-display cameras, addressing complex spatial degradations.

Contribution

The paper introduces UCMNet, a novel uncertainty-aware network that guides adaptive restoration of high-frequency details in UDC images, outperforming prior methods with fewer parameters.

Findings

UCMNet achieves state-of-the-art results on multiple benchmarks.

UCMNet uses uncertainty maps to guide context retrieval for non-uniform degradation.

UCMNet requires 30% fewer parameters than previous models.

Abstract

Under-display cameras (UDCs) allow for full-screen designs by positioning the imaging sensor underneath the display. Nonetheless, light diffraction and scattering through the various display layers result in spatially varying and complex degradations, which significantly reduce high-frequency details. Current PSF-based physical modeling techniques and frequency-separation networks are effective at reconstructing low-frequency structures and maintaining overall color consistency. However, they still face challenges in recovering fine details when dealing with complex, spatially varying degradation. To solve this problem, we propose a lightweight \textbf{U}ncertainty-aware \textbf{C}ontext-\textbf{M}emory \textbf{Network} (\textbf{UCMNet}), for UDC image restoration. Unlike previous methods that apply uniform restoration, UCMNet performs uncertainty-aware adaptive processing to restore high-frequency details in regions with varying degradations. The estimated uncertainty maps, learned through an uncertainty-driven loss, quantify spatial uncertainty induced by diffraction and scattering, and guide the Memory Bank to retrieve region-adaptive context from the Context Bank. This process enables effective modeling of the non-uniform degradation characteristics inherent to UDC imaging. Leveraging this uncertainty as a prior, UCMNet achieves state-of-the-art performance on multiple benchmarks with 30\% fewer parameters than previous models. Project page: \href{https://kdhrick2222.github.io/projects/UCMNet/}{https://kdhrick2222.github.io/projects/UCMNet}.