Dynamic Attention-Guided Diffusion for Image Super-Resolution

TL;DR

This paper introduces YODA, a dynamic attention-guided diffusion method for image super-resolution that selectively focuses on detail-rich regions, leading to improved quality, stability, and state-of-the-art results across various SR tasks.

Contribution

YODA is the first to incorporate dynamic attention maps into diffusion models for image super-resolution, enhancing focus on important regions during denoising.

Findings

Achieves state-of-the-art PSNR, SSIM, LPIPS scores in face and general SR.

Reduces color shift issues and stabilizes training with small batches.

Improves focus on detail-rich regions, enhancing image quality.

Abstract

Diffusion models in image Super-Resolution (SR) treat all image regions uniformly, which risks compromising the overall image quality by potentially introducing artifacts during denoising of less-complex regions. To address this, we propose ``You Only Diffuse Areas'' (YODA), a dynamic attention-guided diffusion process for image SR. YODA selectively focuses on spatial regions defined by attention maps derived from the low-resolution images and the current denoising time step. This time-dependent targeting enables a more efficient conversion to high-resolution outputs by focusing on areas that benefit the most from the iterative refinement process, i.e., detail-rich objects. We empirically validate YODA by extending leading diffusion-based methods SR3, DiffBIR, and SRDiff. Our experiments demonstrate new state-of-the-art performances in face and general SR tasks across PSNR, SSIM, and LPIPS metrics. As a side effect, we find that YODA reduces color shift issues and stabilizes training with small batches.