Quality Assessment of Image Super-Resolution: Balancing Deterministic and Statistical Fidelity

TL;DR

This paper introduces a new 2D quality assessment framework for image super-resolution that balances deterministic and statistical fidelity, proposing the SRIF index which outperforms existing models.

Contribution

It presents a novel 2D fidelity space analysis for SR images and a combined quality index that effectively merges content-dependent sharpness and texture assessments.

Findings

Traditional SR algorithms favor deterministic fidelity, losing statistical fidelity.

GAN-based SR approaches excel in statistical fidelity but may weaken deterministic fidelity.

The proposed SRIF index outperforms state-of-the-art IQA models on subject-rated datasets.

Abstract

There has been a growing interest in developing image super-resolution (SR) algorithms that convert low-resolution (LR) to higher resolution images, but automatically evaluating the visual quality of super-resolved images remains a challenging problem. Here we look at the problem of SR image quality assessment (SR IQA) in a two-dimensional (2D) space of deterministic fidelity (DF) versus statistical fidelity (SF). This allows us to better understand the advantages and disadvantages of existing SR algorithms, which produce images at different clusters in the 2D space of (DF, SF). Specifically, we observe an interesting trend from more traditional SR algorithms that are typically inclined to optimize for DF while losing SF, to more recent generative adversarial network (GAN) based approaches that by contrast exhibit strong advantages in achieving high SF but sometimes appear weak at maintaining DF. Furthermore, we propose an uncertainty weighting scheme based on content-dependent sharpness and texture assessment that merges the two fidelity measures into an overall quality prediction named the Super Resolution Image Fidelity (SRIF) index, which demonstrates superior performance against state-of-the-art IQA models when tested on subject-rated datasets.