Image Restoration via Multi-domain Learning

TL;DR

This paper introduces a multi-domain learning framework integrated into Transformer architectures for image restoration, effectively handling various degradation phenomena with improved efficiency and performance across ten diverse tasks.

Contribution

It proposes a novel multi-domain Transformer model with a Spatial-Wavelet-Fourier structure and multi-scale learning, addressing complexity and generalization in image restoration.

Findings

Outperforms state-of-the-art methods across ten restoration tasks

Achieves a good balance between restoration quality, model size, and computational efficiency

Demonstrates versatility in handling diverse degradation phenomena

Abstract

Due to adverse atmospheric and imaging conditions, natural images suffer from various degradation phenomena. Consequently, image restoration has emerged as a key solution and garnered substantial attention. Although recent Transformer architectures have demonstrated impressive success across various restoration tasks, their considerable model complexity poses significant challenges for both training and real-time deployment. Furthermore, instead of investigating the commonalities among different degradations, most existing restoration methods focus on modifying Transformer under limited restoration priors. In this work, we first review various degradation phenomena under multi-domain perspective, identifying common priors. Then, we introduce a novel restoration framework, which integrates multi-domain learning into Transformer. Specifically, in Token Mixer, we propose a Spatial-Wavelet-Fourier multi-domain structure that facilitates local-region-global multi-receptive field modeling to replace vanilla self-attention. Additionally, in Feed-Forward Network, we incorporate multi-scale learning to fuse multi-domain features at different resolutions. Comprehensive experimental results across ten restoration tasks, such as dehazing, desnowing, motion deblurring, defocus deblurring, rain streak/raindrop removal, cloud removal, shadow removal, underwater enhancement and low-light enhancement, demonstrate that our proposed model outperforms state-of-the-art methods and achieves a favorable trade-off among restoration performance, parameter size, computational cost and inference latency. The code is available at: https://github.com/deng-ai-lab/SWFormer.