Image Super-Resolution Quality Assessment: Structural Fidelity Versus Statistical Naturalness

TL;DR

This paper introduces a 2D quality assessment framework for super-resolution images, balancing structural fidelity and naturalness, and demonstrates that a simple combination of these measures accurately predicts perceived image quality.

Contribution

It proposes a novel 2D evaluation space for SISR quality assessment and shows that a linear combination of structural and statistical measures effectively predicts image quality.

Findings

Linear combination of measures predicts quality accurately

GAN-based methods favor naturalness over fidelity

Traditional methods prioritize structural fidelity

Abstract

Single image super-resolution (SISR) algorithms reconstruct high-resolution (HR) images with their low-resolution (LR) counterparts. It is desirable to develop image quality assessment (IQA) methods that can not only evaluate and compare SISR algorithms, but also guide their future development. In this paper, we assess the quality of SISR generated images in a two-dimensional (2D) space of structural fidelity versus statistical naturalness. This allows us to observe the behaviors of different SISR algorithms as a tradeoff in the 2D space. Specifically, SISR methods are traditionally designed to achieve high structural fidelity but often sacrifice statistical naturalness, while recent generative adversarial network (GAN) based algorithms tend to create more natural-looking results but lose significantly on structural fidelity. Furthermore, such a 2D evaluation can be easily fused to a scalar quality prediction. Interestingly, we find that a simple linear combination of a straightforward local structural fidelity and a global statistical naturalness measures produce surprisingly accurate predictions of SISR image quality when tested using public subject-rated SISR image datasets. Code of the proposed SFSN model is publicly available at \url{https://github.com/weizhou-geek/SFSN}.