TPGDiff: Hierarchical Triple-Prior Guided Diffusion for Image Restoration

TL;DR

TPGDiff introduces a hierarchical diffusion model guided by multiple priors, including structural, semantic, and degradation priors, to improve unified image restoration across various degradation types.

Contribution

It proposes a novel hierarchical prior-guided diffusion framework that integrates multiple sources of priors for robust, unified image restoration.

Findings

Achieves superior performance on diverse restoration benchmarks.

Demonstrates strong generalization across multiple degradation scenarios.

Effectively reconstructs content in severely degraded regions.

Abstract

All-in-one image restoration aims to address diverse degradation types using a single unified model. Existing methods typically rely on degradation priors to guide restoration, yet often struggle to reconstruct content in severely degraded regions. Although recent works leverage semantic information to facilitate content generation, integrating it into the shallow layers of diffusion models often disrupts spatial structures (\emph{e.g.}, blurring artifacts). To address this issue, we propose a Triple-Prior Guided Diffusion (TPGDiff) network for unified image restoration. TPGDiff incorporates degradation priors throughout the diffusion trajectory, while introducing structural priors into shallow layers and semantic priors into deep layers, enabling hierarchical and complementary prior guidance for image reconstruction. Specifically, we leverage multi-source structural cues as structural priors to capture fine-grained details and guide shallow layers representations. To complement this design, we further develop a distillation-driven semantic extractor that yields robust semantic priors, ensuring reliable high-level guidance at deep layers even under severe degradations. Furthermore, a degradation extractor is employed to learn degradation-aware priors, enabling stage-adaptive control of the diffusion process across all timesteps. Extensive experiments on both single- and multi-degradation benchmarks demonstrate that TPGDiff achieves superior performance and generalization across diverse restoration scenarios. Our project page is: https://leoyjtu.github.io/tpgdiff-project.