SRSR: Enhancing Semantic Accuracy in Real-World Image Super-Resolution with Spatially Re-Focused Text-Conditioning

TL;DR

This paper introduces SRSR, a novel super-resolution framework that improves semantic accuracy and reduces hallucinations in image super-resolution by refining text conditioning with spatial guidance and targeted guidance mechanisms.

Contribution

The paper proposes a new plug-and-play framework with Spatially Re-focused Cross-Attention and Spatially Targeted Classifier-Free Guidance to enhance semantic fidelity in diffusion-based super-resolution.

Findings

Outperforms seven state-of-the-art methods in PSNR and SSIM.

Achieves higher perceptual quality on real-world datasets.

Effectively reduces semantic misalignment and hallucinations.

Abstract

Existing diffusion-based super-resolution approaches often exhibit semantic ambiguities due to inaccuracies and incompleteness in their text conditioning, coupled with the inherent tendency for cross-attention to divert towards irrelevant pixels. These limitations can lead to semantic misalignment and hallucinated details in the generated high-resolution outputs. To address these, we propose a novel, plug-and-play spatially re-focused super-resolution (SRSR) framework that consists of two core components: first, we introduce Spatially Re-focused Cross-Attention (SRCA), which refines text conditioning at inference time by applying visually-grounded segmentation masks to guide cross-attention. Second, we introduce a Spatially Targeted Classifier-Free Guidance (STCFG) mechanism that selectively bypasses text influences on ungrounded pixels to prevent hallucinations. Extensive experiments on both synthetic and real-world datasets demonstrate that SRSR consistently outperforms seven state-of-the-art baselines in standard fidelity metrics (PSNR and SSIM) across all datasets, and in perceptual quality measures (LPIPS and DISTS) on two real-world benchmarks, underscoring its effectiveness in achieving both high semantic fidelity and perceptual quality in super-resolution.