Diffusion Transformer meets Multi-level Wavelet Spectrum for Single Image Super-Resolution

TL;DR

This paper introduces DTWSR, a novel image super-resolution method combining diffusion models, transformers, and multi-level wavelet spectra to improve the consistency and realism of reconstructed images.

Contribution

The paper proposes a Diffusion Transformer model utilizing multi-level wavelet spectra and a pyramid tokenization method for enhanced multiscale frequency relation modeling in super-resolution.

Findings

Achieves superior perceptual quality and fidelity on benchmark datasets.

Effectively captures interrelations among multiscale frequency sub-bands.

Produces more consistent and realistic super-resolved images.

Abstract

Discrete Wavelet Transform (DWT) has been widely explored to enhance the performance of image superresolution (SR). Despite some DWT-based methods improving SR by capturing fine-grained frequency signals, most existing approaches neglect the interrelations among multiscale frequency sub-bands, resulting in inconsistencies and unnatural artifacts in the reconstructed images. To address this challenge, we propose a Diffusion Transformer model based on image Wavelet spectra for SR (DTWSR). DTWSR incorporates the superiority of diffusion models and transformers to capture the interrelations among multiscale frequency sub-bands, leading to a more consistence and realistic SR image. Specifically, we use a Multi-level Discrete Wavelet Transform to decompose images into wavelet spectra. A pyramid tokenization method is proposed which embeds the spectra into a sequence of tokens for transformer model, facilitating to capture features from both spatial and frequency domain. A dual-decoder is designed elaborately to handle the distinct variances in low-frequency and high-frequency sub-bands, without omitting their alignment in image generation. Extensive experiments on multiple benchmark datasets demonstrate the effectiveness of our method, with high performance on both perception quality and fidelity.