Trust but Verify: Adaptive Conditioning for Reference-Based Diffusion Super-Resolution via Implicit Reference Correlation Modeling

TL;DR

Ada-RefSR introduces an adaptive diffusion framework that intelligently leverages reference images for super-resolution, balancing guidance and suppression based on reliability, thus improving robustness and quality in real-world scenarios.

Contribution

The paper proposes Ada-RefSR with AICG, a novel adaptive implicit correlation gating mechanism that dynamically controls reference guidance in diffusion-based super-resolution.

Findings

Achieves a strong balance of fidelity, naturalness, and efficiency.

Robust under varying reference alignment conditions.

Outperforms existing methods on multiple datasets.

Abstract

Recent works have explored reference-based super-resolution (RefSR) to mitigate hallucinations in diffusion-based image restoration. A key challenge is that real-world degradations make correspondences between low-quality (LQ) inputs and reference (Ref) images unreliable, requiring adaptive control of reference usage. Existing methods either ignore LQ-Ref correlations or rely on brittle explicit matching, leading to over-reliance on misleading references or under-utilization of valuable cues. To address this, we propose Ada-RefSR, a single-step diffusion framework guided by a "Trust but Verify" principle: reference information is leveraged when reliable and suppressed otherwise. Its core component, Adaptive Implicit Correlation Gating (AICG), employs learnable summary tokens to distill dominant reference patterns and capture implicit correlations with LQ features. Integrated into the attention backbone, AICG provides lightweight, adaptive regulation of reference guidance, serving as a built-in safeguard against erroneous fusion. Experiments on multiple datasets demonstrate that Ada-RefSR achieves a strong balance of fidelity, naturalness, and efficiency, while remaining robust under varying reference alignment.