SPQE: Structure-and-Perception-Based Quality Evaluation for Image Super-Resolution

TL;DR

This paper introduces SPQE, a novel deep-learning-based metric for image super-resolution quality assessment that balances perceptual and structural quality scores, outperforming existing methods.

Contribution

It proposes a unified quality evaluation metric that adaptively combines perceptual and structural scores with theoretical analysis and deep regressors.

Findings

Outperforms state-of-the-art SR-IQA methods on multiple datasets

Balances perceptual and structural quality assessments effectively

Uses adaptive weights for improved evaluation accuracy

Abstract

The image Super-Resolution (SR) technique has greatly improved the visual quality of images by enhancing their resolutions. It also calls for an efficient SR Image Quality Assessment (SR-IQA) to evaluate those algorithms or their generated images. In this paper, we focus on the SR-IQA under deep learning and propose a Structure-and-Perception-based Quality Evaluation (SPQE). In emerging deep-learning-based SR, a generated high-quality, visually pleasing image may have different structures from its corresponding low-quality image. In such case, how to balance the quality scores between no-reference perceptual quality and referenced structural similarity is a critical issue. To help ease this problem, we give a theoretical analysis on this tradeoff and further calculate adaptive weights for the two types of quality scores. We also propose two deep-learning-based regressors to model the no-reference and referenced scores. By combining the quality scores and their weights, we propose a unified SPQE metric for SR-IQA. Experimental results demonstrate that the proposed method outperforms the state-of-the-arts in different datasets.