AllRestorer: All-in-One Transformer for Image Restoration under Composite Degradations

TL;DR

AllRestorer introduces a Transformer-based framework that adaptively restores images with multiple degradations by modeling all impairments simultaneously, outperforming existing methods that rely on scene descriptors.

Contribution

The paper proposes AllRestorer, a novel all-in-one Transformer model that adaptively handles multiple degradations without scene descriptor misguidance, using a new AiOTB module with composite scene descriptors.

Findings

Achieves a 5.00 dB PSNR increase on CDD-11 dataset.

Effectively models relationships between degradations and image embeddings.

Avoids errors from scene descriptor inaccuracies.

Abstract

Image restoration models often face the simultaneous interaction of multiple degradations in real-world scenarios. Existing approaches typically handle single or composite degradations based on scene descriptors derived from text or image embeddings. However, due to the varying proportions of different degradations within an image, these scene descriptors may not accurately differentiate between degradations, leading to suboptimal restoration in practical applications. To address this issue, we propose a novel Transformer-based restoration framework, AllRestorer. In AllRestorer, we enable the model to adaptively consider all image impairments, thereby avoiding errors from scene descriptor misdirection. Specifically, we introduce an All-in-One Transformer Block (AiOTB), which adaptively removes all degradations present in a given image by modeling the relationships between all degradations and the image embedding in latent space. To accurately address different variations potentially present within the same type of degradation and minimize ambiguity, AiOTB utilizes a composite scene descriptor consisting of both image and text embeddings to define the degradation. Furthermore, AiOTB includes an adaptive weight for each degradation, allowing for precise control of the restoration intensity. By leveraging AiOTB, AllRestorer avoids misdirection caused by inaccurate scene descriptors, achieving a 5.00 dB increase in PSNR compared to the baseline on the CDD-11 dataset.