Co-learning Single-Step Diffusion Upsampler and Downsampler with Two Discriminators and Distillation

TL;DR

This paper introduces a co-learning framework for super-resolution that jointly trains a diffusion-based upsampler and a learnable downsampler with discriminators and distillation, achieving state-of-the-art results in real-world and face SR tasks.

Contribution

It presents a novel joint optimization of upsampler and downsampler with discriminators and cyclic distillation, improving realism and structural preservation in super-resolution.

Findings

Achieves state-of-the-art performance on real-world and face SR tasks.

Generates diverse LR-HR pairs for robust training.

Enhances efficiency with a single inference step.

Abstract

Super-resolution (SR) aims to reconstruct high-resolution (HR) images from their low-resolution (LR) counterparts, often relying on effective downsampling to generate diverse and realistic training pairs. In this work, we propose a co-learning framework that jointly optimizes a single-step diffusion-based upsampler and a learnable downsampler, enhanced by two discriminators and a cyclic distillation strategy. Our learnable downsampler is designed to better capture realistic degradation patterns while preserving structural details in the LR domain, which is crucial for enhancing SR performance. By leveraging a diffusion-based approach, our model generates diverse LR-HR pairs during training, enabling robust learning across varying degradations. We demonstrate the effectiveness of our method on both general real-world and domain-specific face SR tasks, achieving state-of-the-art performance in both fidelity and perceptual quality. Our approach not only improves efficiency with a single inference step but also ensures high-quality image reconstruction, bridging the gap between synthetic and real-world SR scenarios.