Your Pre-trained Diffusion Model Secretly Knows Restoration

TL;DR

This paper reveals that pre-trained diffusion models inherently possess restoration capabilities that can be unlocked through prompt learning, enabling effective image and video restoration without fine-tuning or additional modules.

Contribution

The authors demonstrate that direct prompt learning within a diffusion bridge framework unlocks inherent restoration behavior in pre-trained diffusion models, avoiding fine-tuning.

Findings

Achieves competitive restoration performance across diverse degradations.

Enables high-quality restoration without fine-tuning or control modules.

Extends approach to both image and video diffusion models.

Abstract

Pre-trained diffusion models have enabled significant advancements in All-in-One Restoration (AiOR), offering improved perceptual quality and generalization. However, diffusion-based restoration methods primarily rely on fine-tuning or Control-Net style modules to leverage the pre-trained diffusion model's priors for AiOR. In this work, we show that these pre-trained diffusion models inherently possess restoration behavior, which can be unlocked by directly learning prompt embeddings at the output of the text encoder. Interestingly, this behavior is largely inaccessible through text prompts and text-token embedding optimization. Furthermore, we observe that naive prompt learning is unstable because the forward noising process using degraded images is misaligned with the reverse sampling trajectory. To resolve this, we train prompts within a diffusion bridge formulation that aligns training and inference dynamics, enforcing a coherent denoising path from noisy degraded states to clean images. Building on these insights, we introduce our lightweight learned prompts on the pre-trained WAN video model and FLUX image models, converting them into high-performing restoration models. Extensive experiments demonstrate that our approach achieves competitive performance and generalization across diverse degradations, while avoiding fine-tuning and restoration-specific control modules.