Learning Dual Transformers for All-In-One Image Restoration from a Frequency Perspective

TL;DR

This paper introduces a dual-transformer framework that leverages frequency analysis to adaptively restore images affected by diverse degradations, achieving superior performance across multiple restoration tasks.

Contribution

The paper presents a novel frequency-aware dual-transformer model with degradation estimation and adaptive restoration components for all-in-one image restoration.

Findings

Outperforms existing methods in five restoration tasks

Effective in real-world and spatially variant degradations

Handles unseen degradation levels successfully

Abstract

This work aims to tackle the all-in-one image restoration task, which seeks to handle multiple types of degradation with a single model. The primary challenge is to extract degradation representations from the input degraded images and use them to guide the model's adaptation to specific degradation types. Building on the insight that various degradations affect image content differently across frequency bands, we propose a new dual-transformer approach comprising two components: a frequency-aware Degradation estimation transformer (Dformer) and a degradation-adaptive Restoration transformer (Rformer). The Dformer captures the essential characteristics of various degradations by decomposing the input into different frequency components. By understanding how degradations affect these frequency components, the Dformer learns robust priors that effectively guide the restoration process. The Rformer then employs a degradation-adaptive self-attention module to selectively focus on the most affected frequency components, guided by the learned degradation representations. Extensive experimental results demonstrate that our approach outperforms existing methods in five representative restoration tasks, including denoising, deraining, dehazing, deblurring, and low-light enhancement. Additionally, our method offers benefits for handling, real-world degradations, spatially variant degradations, and unseen degradation levels.