Neural Discrimination-Prompted Transformers for Efficient UHD Image Restoration and Enhancement

TL;DR

UHDPromer is a novel neural network that leverages neural discrimination priors within a Transformer architecture to efficiently restore and enhance ultra-high-definition images across various tasks.

Contribution

The paper introduces Neural Discrimination Priors and integrates them into a Transformer-based model for UHD image restoration, achieving state-of-the-art results with high efficiency.

Findings

Achieves state-of-the-art performance on UHD image restoration tasks.

Maintains high computational efficiency compared to existing methods.

Effectively handles low-light, dehazing, and deblurring tasks.

Abstract

We propose a simple yet effective UHDPromer, a neural discrimination-prompted Transformer, for Ultra-High-Definition (UHD) image restoration and enhancement. Our UHDPromer is inspired by an interesting observation that there implicitly exist neural differences between high-resolution and low-resolution features, and exploring such differences can facilitate low-resolution feature representation. To this end, we first introduce Neural Discrimination Priors (NDP) to measure the differences and then integrate NDP into the proposed Neural Discrimination-Prompted Attention (NDPA) and Neural Discrimination-Prompted Network (NDPN). The proposed NDPA re-formulates the attention by incorporating NDP to globally perceive useful discrimination information, while the NDPN explores a continuous gating mechanism guided by NDP to selectively permit the passage of beneficial content. To enhance the quality of restored images, we propose a super-resolution-guided reconstruction approach, which is guided by super-resolving low-resolution features to facilitate final UHD image restoration. Experiments show that UHDPromer achieves the best computational efficiency while still maintaining state-of-the-art performance on $3$ UHD image restoration and enhancement tasks, including low-light image enhancement, image dehazing, and image deblurring. The source codes and pre-trained models will be made available at https://github.com/supersupercong/uhdpromer.