Elucidating and Endowing the Diffusion Training Paradigm for General Image Restoration

TL;DR

This paper adapts diffusion training paradigms for general image restoration, introducing new regularization strategies and incremental training to improve single-task and multi-task IR performance.

Contribution

It systematically analyzes diffusion training principles for IR and proposes a novel framework with regularization and task-specific adaptors for enhanced generalization.

Findings

Improved IR network generalization in single-task scenarios.

Superior multi-task IR performance with the proposed framework.

Seamless integration into existing IR architectures.

Abstract

While diffusion models demonstrate strong generative capabilities in image restoration (IR) tasks, their complex architectures and iterative processes limit their practical application compared to mainstream reconstruction-based general ordinary IR networks. Existing approaches primarily focus on optimizing network architecture and diffusion paths but overlook the integration of the diffusion training paradigm within general ordinary IR frameworks. To address these challenges, this paper elucidates key principles for adapting the diffusion training paradigm to general IR training through systematic analysis of time-step dependencies, network hierarchies, noise-level relationships, and multi-restoration task correlations, proposing a new IR framework supported by diffusion-based training. To enable IR networks to simultaneously restore images and model generative representations, we introduce a series of regularization strategies that align diffusion objectives with IR tasks, improving generalization in single-task scenarios. Furthermore, recognizing that diffusion-based generation exerts varying influences across different IR tasks, we develop an incremental training paradigm and task-specific adaptors, further enhancing performance in multi-task unified IR. Experiments demonstrate that our method significantly improves the generalization of IR networks in single-task IR and achieves superior performance in multi-task unified IR. Notably, the proposed framework can be seamlessly integrated into existing general IR architectures.