Fourier Space Losses for Efficient Perceptual Image Super-Resolution

TL;DR

This paper introduces Fourier space loss functions and a Fourier domain discriminator to enhance perceptual image super-resolution in efficient models, significantly improving quality while maintaining high speed and low complexity.

Contribution

It proposes novel Fourier space and combined spatial-frequency domain loss functions, along with a Fourier domain discriminator, to boost perceptual quality in efficient super-resolution models.

Findings

Improved high-frequency content restoration in super-resolution.

Achieved comparable perceptual quality with significantly faster models.

Enhanced training guidance through Fourier domain supervision.

Abstract

Many super-resolution (SR) models are optimized for high performance only and therefore lack efficiency due to large model complexity. As large models are often not practical in real-world applications, we investigate and propose novel loss functions, to enable SR with high perceptual quality from much more efficient models. The representative power for a given low-complexity generator network can only be fully leveraged by strong guidance towards the optimal set of parameters. We show that it is possible to improve the performance of a recently introduced efficient generator architecture solely with the application of our proposed loss functions. In particular, we use a Fourier space supervision loss for improved restoration of missing high-frequency (HF) content from the ground truth image and design a discriminator architecture working directly in the Fourier domain to better match the target HF distribution. We show that our losses' direct emphasis on the frequencies in Fourier-space significantly boosts the perceptual image quality, while at the same time retaining high restoration quality in comparison to previously proposed loss functions for this task. The performance is further improved by utilizing a combination of spatial and frequency domain losses, as both representations provide complementary information during training. On top of that, the trained generator achieves comparable results with and is 2.4x and 48x faster than state-of-the-art perceptual SR methods RankSRGAN and SRFlow respectively.