One-Step Diffusion-based Real-World Image Super-Resolution with Visual Perception Distillation

TL;DR

This paper introduces VPD-SR, a one-step diffusion-based super-resolution framework that combines semantic and high-frequency perception guidance to produce high-quality, perceptually faithful images efficiently.

Contribution

The paper proposes a novel one-step diffusion model for super-resolution that integrates semantic supervision and high-frequency perception loss, improving perceptual quality and semantic fidelity.

Findings

VPD-SR outperforms previous methods in perceptual quality.

Achieves comparable or better results with just one-step sampling.

Enhances semantic consistency and high-frequency detail restoration.

Abstract

Diffusion-based models have been widely used in various visual generation tasks, showing promising results in image super-resolution (SR), while typically being limited by dozens or even hundreds of sampling steps. Although existing methods aim to accelerate the inference speed of multi-step diffusion-based SR methods through knowledge distillation, their generated images exhibit insufficient semantic alignment with real images, resulting in suboptimal perceptual quality reconstruction, specifically reflected in the CLIPIQA score. These methods still have many challenges in perceptual quality and semantic fidelity. Based on the challenges, we propose VPD-SR, a novel visual perception diffusion distillation framework specifically designed for SR, aiming to construct an effective and efficient one-step SR model. Specifically, VPD-SR consists of two components: Explicit Semantic-aware Supervision (ESS) and High-Frequency Perception (HFP) loss. Firstly, the ESS leverages the powerful visual perceptual understanding capabilities of the CLIP model to extract explicit semantic supervision, thereby enhancing semantic consistency. Then, Considering that high-frequency information contributes to the visual perception quality of images, in addition to the vanilla distillation loss, the HFP loss guides the student model to restore the missing high-frequency details in degraded images that are critical for enhancing perceptual quality. Lastly, we expand VPD-SR in adversarial training manner to further enhance the authenticity of the generated content. Extensive experiments conducted on synthetic and real-world datasets demonstrate that the proposed VPD-SR achieves superior performance compared to both previous state-of-the-art methods and the teacher model with just one-step sampling.