Multi-Stage Progressive Image Restoration

TL;DR

This paper introduces MPRNet, a multi-stage progressive architecture for image restoration that balances local details and contextual information through staged learning, adaptive attention, and extensive information exchange, achieving state-of-the-art results.

Contribution

The paper presents a novel multi-stage architecture with adaptive per-pixel attention and bidirectional information exchange, improving image restoration performance across multiple tasks.

Findings

Significant performance improvements on ten datasets.

Effective balancing of local details and contextual features.

Versatile application to deraining, deblurring, and denoising.

Abstract

Image restoration tasks demand a complex balance between spatial details and high-level contextualized information while recovering images. In this paper, we propose a novel synergistic design that can optimally balance these competing goals. Our main proposal is a multi-stage architecture, that progressively learns restoration functions for the degraded inputs, thereby breaking down the overall recovery process into more manageable steps. Specifically, our model first learns the contextualized features using encoder-decoder architectures and later combines them with a high-resolution branch that retains local information. At each stage, we introduce a novel per-pixel adaptive design that leverages in-situ supervised attention to reweight the local features. A key ingredient in such a multi-stage architecture is the information exchange between different stages. To this end, we propose a two-faceted approach where the information is not only exchanged sequentially from early to late stages, but lateral connections between feature processing blocks also exist to avoid any loss of information. The resulting tightly interlinked multi-stage architecture, named as MPRNet, delivers strong performance gains on ten datasets across a range of tasks including image deraining, deblurring, and denoising. The source code and pre-trained models are available at https://github.com/swz30/MPRNet.