Assessing invariance to affine transformations in image quality metrics

TL;DR

This paper introduces a methodology to evaluate image quality metrics based on their invariance to affine transformations, highlighting that current metrics lack human-like invisibility thresholds and suggesting the need for models that better mimic human perception.

Contribution

The paper proposes a novel method to assess and compare image quality metrics' invariance to affine transformations using a common subjective representation and psychophysical thresholds.

Findings

Existing metrics do not exhibit human-like invisibility thresholds.

The methodology can be applied to any image quality metric.

Current models may overlook important invariances in human vision.

Abstract

Subjective image quality metrics are usually evaluated according to the correlation with human opinion in databases with distortions that may appear in digital media. However, these oversee affine transformations which may represent better the changes in the images actually happening in natural conditions. Humans can be particularly invariant to these natural transformations, as opposed to the digital ones. In this work, we propose a methodology to evaluate any image quality metric by assessing their invariance to affine transformations, specifically: rotation, translation, scaling, and changes in spectral illumination. Here, invariance refers to the fact that certain distances should be neglected if their values are below a threshold. This is what we call invisibility threshold of a metric. Our methodology consists of two elements: (1) the determination of a visibility threshold in a subjective representation common to every metric, and (2) a transduction from the distance values of the metric and this common representation. This common representation is based on subjective ratings of readily available image quality databases. We determine the threshold in such common representation (the first element) using accurate psychophysics. Then, the transduction (the second element) can be trivially fitted for any metric: with the provided threshold extension of the method to any metric is straightforward. We test our methodology with some well-established metrics and find that none of them show human-like invisibility thresholds. This means that tuning the models exclusively to predict the visibility of generic distortions may disregard other properties of human vision as for instance invariances or invisibility thresholds. The data and code are publicly available to test other metrics.