ConStyle v2: A Strong Prompter for All-in-One Image Restoration

TL;DR

ConStyle v2 is a versatile, plug-and-play prompter that significantly enhances various U-Net based image restoration models to handle multiple degradations effectively without fine-tuning.

Contribution

The paper introduces ConStyle v2, a strong, easy-to-train prompter that improves all-in-one image restoration capabilities across different model architectures.

Findings

ConStyle v2 improves performance on multiple degradation types.

It enhances various architectures like Restormer, NAFNet, and MAXIM-1S.

Training requires less than two GPU days without fine-tuning.

Abstract

This paper introduces ConStyle v2, a strong plug-and-play prompter designed to output clean visual prompts and assist U-Net Image Restoration models in handling multiple degradations. The joint training process of IRConStyle, an Image Restoration framework consisting of ConStyle and a general restoration network, is divided into two stages: first, pre-training ConStyle alone, and then freezing its weights to guide the training of the general restoration network. Three improvements are proposed in the pre-training stage to train ConStyle: unsupervised pre-training, adding a pretext task (i.e. classification), and adopting knowledge distillation. Without bells and whistles, we can get ConStyle v2, a strong prompter for all-in-one Image Restoration, in less than two GPU days and doesn't require any fine-tuning. Extensive experiments on Restormer (transformer-based), NAFNet (CNN-based), MAXIM-1S (MLP-based), and a vanilla CNN network demonstrate that ConStyle v2 can enhance any U-Net style Image Restoration models to all-in-one Image Restoration models. Furthermore, models guided by the well-trained ConStyle v2 exhibit superior performance in some specific degradation compared to ConStyle.