Diffusion Once and Done: Degradation-Aware LoRA for Efficient All-in-One Image Restoration

TL;DR

This paper introduces Diffusion Once and Done (DOD), an efficient all-in-one image restoration method using a single-step diffusion process, multi-degradation feature modulation, and parameter-efficient adaptation to improve quality and speed.

Contribution

The paper proposes a novel one-step diffusion-based image restoration method with multi-degradation feature modulation and low-rank adaptation for better performance and efficiency.

Findings

Outperforms existing diffusion-based methods in visual quality

Achieves faster inference with only one-step sampling

Effectively adapts to diverse degradation types

Abstract

Diffusion models have revealed powerful potential in all-in-one image restoration (AiOIR), which is talented in generating abundant texture details. The existing AiOIR methods either retrain a diffusion model or fine-tune the pretrained diffusion model with extra conditional guidance. However, they often suffer from high inference costs and limited adaptability to diverse degradation types. In this paper, we propose an efficient AiOIR method, Diffusion Once and Done (DOD), which aims to achieve superior restoration performance with only one-step sampling of Stable Diffusion (SD) models. Specifically, multi-degradation feature modulation is first introduced to capture different degradation prompts with a pretrained diffusion model. Then, parameter-efficient conditional low-rank adaptation integrates the prompts to enable the fine-tuning of the SD model for adapting to different degradation types. Besides, a high-fidelity detail enhancement module is integrated into the decoder of SD to improve structural and textural details. Experiments demonstrate that our method outperforms existing diffusion-based restoration approaches in both visual quality and inference efficiency.