Correlation Matching Transformation Transformers for UHD Image Restoration

TL;DR

UHDformer is a Transformer-based model designed for UHD image restoration, utilizing dual learning spaces and novel modules to enhance feature representation and achieve high performance with significantly reduced model size.

Contribution

The paper introduces UHDformer with dual high- and low-resolution learning spaces and novel modules DualCMT and ACM for improved feature transformation and modulation.

Findings

Reduces model size by about 97% compared to state-of-the-art methods.

Significantly improves performance on UHD image restoration tasks.

Effective in low-light enhancement, dehazing, and deblurring.

Abstract

This paper proposes UHDformer, a general Transformer for Ultra-High-Definition (UHD) image restoration. UHDformer contains two learning spaces: (a) learning in high-resolution space and (b) learning in low-resolution space. The former learns multi-level high-resolution features and fuses low-high features and reconstructs the residual images, while the latter explores more representative features learning from the high-resolution ones to facilitate better restoration. To better improve feature representation in low-resolution space, we propose to build feature transformation from the high-resolution space to the low-resolution one. To that end, we propose two new modules: Dual-path Correlation Matching Transformation module (DualCMT) and Adaptive Channel Modulator (ACM). The DualCMT selects top C/r (r is greater or equal to 1 which controls the squeezing level) correlation channels from the max-pooling/mean-pooling high-resolution features to replace low-resolution ones in Transformers, which can effectively squeeze useless content to improve the feature representation in low-resolution space to facilitate better recovery. The ACM is exploited to adaptively modulate multi-level high-resolution features, enabling to provide more useful features to low-resolution space for better learning. Experimental results show that our UHDformer reduces about ninety-seven percent model sizes compared with most state-of-the-art methods while significantly improving performance under different training sets on 3 UHD image restoration tasks, including low-light image enhancement, image dehazing, and image deblurring. The source codes will be made available at https://github.com/supersupercong/UHDformer.