StructSR: Refuse Spurious Details in Real-World Image Super-Resolution

TL;DR

StructSR is a plug-and-play method that enhances structural fidelity and suppresses spurious details in diffusion-based real-world image super-resolution, significantly improving image quality without extra training or external priors.

Contribution

It introduces the Structure-Aware Screening mechanism for real-time structural fidelity enhancement in diffusion-based super-resolution models.

Findings

Improves PSNR and SSIM by over 5% and 9% on synthetic datasets.

Enhances image structure and texture fidelity across multiple diffusion-based methods.

Operates without additional fine-tuning or external priors.

Abstract

Diffusion-based models have shown great promise in real-world image super-resolution (Real-ISR), but often generate content with structural errors and spurious texture details due to the empirical priors and illusions of these models. To address this issue, we introduce StructSR, a simple, effective, and plug-and-play method that enhances structural fidelity and suppresses spurious details for diffusion-based Real-ISR. StructSR operates without the need for additional fine-tuning, external model priors, or high-level semantic knowledge. At its core is the Structure-Aware Screening (SAS) mechanism, which identifies the image with the highest structural similarity to the low-resolution (LR) input in the early inference stage, allowing us to leverage it as a historical structure knowledge to suppress the generation of spurious details. By intervening in the diffusion inference process, StructSR seamlessly integrates with existing diffusion-based Real-ISR models. Our experimental results demonstrate that StructSR significantly improves the fidelity of structure and texture, improving the PSNR and SSIM metrics by an average of 5.27% and 9.36% on a synthetic dataset (DIV2K-Val) and 4.13% and 8.64% on two real-world datasets (RealSR and DRealSR) when integrated with four state-of-the-art diffusion-based Real-ISR methods.