Degradation-Aware Feature Perturbation for All-in-One Image Restoration

TL;DR

This paper introduces DFPIR, a novel all-in-one image restoration model that uses degradation-aware feature perturbations to improve performance across multiple restoration tasks by aligning feature space with degradation types.

Contribution

The paper proposes a new degradation-aware feature perturbation method and a dedicated perturbation block to enhance a unified model for diverse image restoration tasks.

Findings

Achieves state-of-the-art results on multiple image restoration benchmarks.

Effectively aligns feature space with degradation types, reducing task interference.

Demonstrates versatility across tasks like denoising, dehazing, deraining, and deblurring.

Abstract

All-in-one image restoration aims to recover clear images from various degradation types and levels with a unified model. Nonetheless, the significant variations among degradation types present challenges for training a universal model, often resulting in task interference, where the gradient update directions of different tasks may diverge due to shared parameters. To address this issue, motivated by the routing strategy, we propose DFPIR, a novel all-in-one image restorer that introduces Degradation-aware Feature Perturbations(DFP) to adjust the feature space to align with the unified parameter space. In this paper, the feature perturbations primarily include channel-wise perturbations and attention-wise perturbations. Specifically, channel-wise perturbations are implemented by shuffling the channels in high-dimensional space guided by degradation types, while attention-wise perturbations are achieved through selective masking in the attention space. To achieve these goals, we propose a Degradation-Guided Perturbation Block (DGPB) to implement these two functions, positioned between the encoding and decoding stages of the encoder-decoder architecture. Extensive experimental results demonstrate that DFPIR achieves state-of-the-art performance on several all-in-one image restoration tasks including image denoising, image dehazing, image deraining, motion deblurring, and low-light image enhancement. Our codes are available at https://github.com/TxpHome/DFPIR.