Zero-shot Adaptation of Stable Diffusion via Plug-in Hierarchical Degradation Representation for Real-World Super-Resolution

TL;DR

This paper introduces HD-CLIP, a plug-and-play hierarchical degradation representation that enhances zero-shot real-world super-resolution by guiding diffusion models with semantic and degradation cues, improving detail and realism.

Contribution

The paper proposes HD-CLIP, a novel hierarchical degradation embedding that captures ordered degradation levels and integrates seamlessly into diffusion models for improved zero-shot super-resolution.

Findings

HD-CLIP improves detail fidelity in super-resolution tasks.

The method enhances perceptual realism across diverse datasets.

It effectively suppresses artifacts and hallucinations.

Abstract

Real-World Image Super-Resolution (Real-ISR) aims to recover high-quality images from low-quality inputs degraded by unknown and complex real-world factors. Real-world scenarios involve diverse and coupled degradations, making it necessary to provide diffusion models with richer and more informative guidance. However, existing methods often assume known degradation severity and rely on CLIP text encoders that cannot capture numerical severity, limiting their generalization ability. To address this, we propose \textbf{HD-CLIP} (\textbf{H}ierarchical \textbf{D}egradation CLIP), which decomposes a low-quality image into a semantic embedding and an ordinal degradation embedding that captures ordered relationships and allows interpolation across unseen levels. Furthermore, we integrated it into diffusion models via classifier-free guidance (CFG) and proposed classifier-free projection guidance (CFPG). HD-CLIP leverages semantic cues to guide generative restoration while using degradation cues to suppress undesired hallucinations and artifacts. As a \textbf{plug-and-play module}, HD-CLIP can be seamlessly integrated into various super-resolution frameworks without training, significantly improving detail fidelity and perceptual realism across diverse real-world datasets.