Frequency Domain-based Perceptual Loss for Super Resolution

TL;DR

This paper proposes Frequency Domain Perceptual Loss (FDPL), a novel loss function for super resolution that operates in the frequency domain to better align with human perception, leading to improved PSNR and qualitative results.

Contribution

FDPL is the first perceptual loss function computed in the frequency domain, emphasizing frequencies relevant to human perception for super resolution tasks.

Findings

Higher average PSNR (30.94 vs. 30.59) with FDPL

Improved qualitative perceptual results

Correlation between FDPL reduction and PSNR increase

Abstract

We introduce Frequency Domain Perceptual Loss (FDPL), a loss function for single image super resolution (SR). Unlike previous loss functions used to train SR models, which are all calculated in the pixel (spatial) domain, FDPL is computed in the frequency domain. By working in the frequency domain we can encourage a given model to learn a mapping that prioritizes those frequencies most related to human perception. While the goal of FDPL is not to maximize the Peak Signal to Noise Ratio (PSNR), we found that there is a correlation between decreasing FDPL and increasing PSNR. Training a model with FDPL results in a higher average PSRN (30.94), compared to the same model trained with pixel loss (30.59), as measured on the Set5 image dataset. We also show that our method achieves higher qualitative results, which is the goal of a perceptual loss function. However, it is not clear that the improved perceptual quality is due to the slightly higher PSNR or the perceptual nature of FDPL.