Training Generative Image Super-Resolution Models by Wavelet-Domain Losses Enables Better Control of Artifacts

TL;DR

This paper introduces wavelet-domain loss functions for training GAN-based super-resolution models, enabling better control over artifacts and hallucinations by focusing on high-frequency details in the wavelet domain.

Contribution

It is the first to apply wavelet-domain losses specifically to super-resolution training, improving the distinction between genuine details and artifacts compared to previous RGB or Fourier-based methods.

Findings

Improved perception-distortion trade-off in super-resolution tasks.

Better artifact control and detail preservation demonstrated visually.

Wavelet-domain discriminator enhances high-frequency feature learning.

Abstract

Super-resolution (SR) is an ill-posed inverse problem, where the size of the set of feasible solutions that are consistent with a given low-resolution image is very large. Many algorithms have been proposed to find a "good" solution among the feasible solutions that strike a balance between fidelity and perceptual quality. Unfortunately, all known methods generate artifacts and hallucinations while trying to reconstruct high-frequency (HF) image details. A fundamental question is: Can a model learn to distinguish genuine image details from artifacts? Although some recent works focused on the differentiation of details and artifacts, this is a very challenging problem and a satisfactory solution is yet to be found. This paper shows that the characterization of genuine HF details versus artifacts can be better learned by training GAN-based SR models using wavelet-domain loss functions compared to RGB-domain or Fourier-space losses. Although wavelet-domain losses have been used in the literature before, they have not been used in the context of the SR task. More specifically, we train the discriminator only on the HF wavelet sub-bands instead of on RGB images and the generator is trained by a fidelity loss over wavelet subbands to make it sensitive to the scale and orientation of structures. Extensive experimental results demonstrate that our model achieves better perception-distortion trade-off according to multiple objective measures and visual evaluations.