Ambiguity of Objective Image Quality Metrics: A New Methodology for Performance Evaluation

TL;DR

This paper introduces a new methodology to evaluate the ambiguity of objective image quality metrics by defining ambiguity intervals, considering viewing conditions, and demonstrating their application across multiple metrics and databases.

Contribution

It proposes a novel approach to quantify the ambiguity of image quality metrics using ambiguity intervals that account for viewing conditions, enhancing performance evaluation.

Findings

Ambiguity intervals vary across metrics and viewing distances.

Ambiguity intervals can supplement traditional performance metrics.

Viewing distance influences the size of ambiguity intervals.

Abstract

Objective image quality metrics try to estimate the perceptual quality of the given image by considering the characteristics of the human visual system. However, it is possible that the metrics produce different quality scores even for two images that are perceptually indistinguishable by human viewers, which have not been considered in the existing studies related to objective quality assessment. In this paper, we address the issue of ambiguity of objective image quality assessment. We propose an approach to obtain an ambiguity interval of an objective metric, within which the quality score difference is not perceptually significant. In particular, we use the visual difference predictor, which can consider viewing conditions that are important for visual quality perception. In order to demonstrate the usefulness of the proposed approach, we conduct experiments with 33 state-of-the-art image quality metrics in the viewpoint of their accuracy and ambiguity for three image quality databases. The results show that the ambiguity intervals can be applied as an additional figure of merit when conventional performance measurement does not determine superiority between the metrics. The effect of the viewing distance on the ambiguity interval is also shown.