A HVS-inspired Attention to Improve Loss Metrics for CNN-based Perception-Oriented Super-Resolution

TL;DR

This paper introduces a human visual system-inspired spatial attention mechanism that enhances perceptual loss functions in CNN-based super-resolution, leading to more natural and perceptually important image reconstructions.

Contribution

It proposes a novel spatial attention mechanism based on human contrast sensitivity to improve perceptual loss functions in super-resolution CNNs.

Findings

Improved perceptual quality of super-resolved images.

Enhanced ability of loss functions to focus on perceptually important regions.

More natural image reconstructions achieved.

Abstract

Deep Convolutional Neural Network (CNN) features have been demonstrated to be effective perceptual quality features. The perceptual loss, based on feature maps of pre-trained CNN's has proven to be remarkably effective for CNN based perceptual image restoration problems. In this work, taking inspiration from the the Human Visual System (HVS) and visual perception, we propose a spatial attention mechanism based on the dependency human contrast sensitivity on spatial frequency. We identify regions in input images, based on the underlying spatial frequency, which are not generally well reconstructed during Super-Resolution but are most important in terms of visual sensitivity. Based on this prior, we design a spatial attention map that is applied to feature maps in the perceptual loss and its variants, helping them to identify regions that are of more perceptual importance. The results demonstrate the our technique improves the ability of the perceptual loss and contextual loss to deliver more natural images in CNN based super-resolution.