Restore Anything with Masks: Leveraging Mask Image Modeling for Blind All-in-One Image Restoration

TL;DR

This paper introduces RAM, a mask image modeling approach for all-in-one blind image restoration, which improves content extraction and achieves state-of-the-art results across multiple degradation types.

Contribution

The paper presents a novel two-stage pipeline with masked pre-training and selective fine-tuning using Mask Attribute Conductance, enhancing all-in-one image restoration performance.

Findings

Achieves state-of-the-art results on multiple restoration tasks

Enhances content priors extraction across degradations

Selective fine-tuning improves model efficiency and performance

Abstract

All-in-one image restoration aims to handle multiple degradation types using one model. This paper proposes a simple pipeline for all-in-one blind image restoration to Restore Anything with Masks (RAM). We focus on the image content by utilizing Mask Image Modeling to extract intrinsic image information rather than distinguishing degradation types like other methods. Our pipeline consists of two stages: masked image pre-training and fine-tuning with mask attribute conductance. We design a straightforward masking pre-training approach specifically tailored for all-in-one image restoration. This approach enhances networks to prioritize the extraction of image content priors from various degradations, resulting in a more balanced performance across different restoration tasks and achieving stronger overall results. To bridge the gap of input integrity while preserving learned image priors as much as possible, we selectively fine-tuned a small portion of the layers. Specifically, the importance of each layer is ranked by the proposed Mask Attribute Conductance (MAC), and the layers with higher contributions are selected for finetuning. Extensive experiments demonstrate that our method achieves state-of-the-art performance. Our code and model will be released at \href{https://github.com/Dragonisss/RAM}{https://github.com/Dragonisss/RAM}.