Review of Measures Used for Evaluating Color Difference Models

TL;DR

This paper reviews various measures for evaluating color difference models, analyzing their properties, equivalences, and coordinate independence, and introduces insights into their optimization and applicability.

Contribution

It provides a comprehensive comparison of color difference measures, highlighting the coordinate independence of certain measures and proposing improvements for their evaluation.

Findings

All measures are equivalent for small deviations.

Only STRESS is coordinate independent.

γ-1 measures allow simple global optimization.

Abstract

We made a detailed review of the difference measures which have been used to judge the differences between experimentally determined color differences and theoretically defined ones, so-called line elements, for the human visual system. To eliminate the statistical errors due to variable and usually arbitrary sampling of the directions in a color point, we integrate the measures over a complete ellipsoid/ellipse. It turns out that in the limit for small deviations from circularity all proposed measures ($V_{AB}$, $\gamma-1$, $CV$ and $\mathrm{STRESS}$) are equivalent. For greater deviations the measures become distinct with $\gamma-1$ the most sensitive and $\mathrm{STRESS}$ the least. Ideally a difference measure should be coordinate independent and then it is advantageous to apply an affine transformation to both sets, e.g. turning the theoretical one into the unit ball. Although MacAdam already used this method but sampled the transformed ellipse, we integrate over the ellipsoid/ellipse. Comparing the results with the base measures we show that only $\mathrm{STRESS}$ is coordinate independent. Judging whether a single ellipsoid/ellipse resembles a unit ball can easily be done by comparing the eigenvalues with one and we show that our previously proposed error measure $d_{ev}$ (Candry e.a. Optics Express, 30, 36307, 2022) is the eigenvalue version of $\gamma-1$. We show why the short lived correlation coefficient $r$ was justly abandoned, being very coordinate dependent, but that Pant's recent geometric measure $1-R$ on the other hand is coordinate independent. All measures are routinely made scale invariant by the introduction of a scaling parameter, to be optimized. Lastly we show that from all measures the $\gamma-1$ ones are the only ones permitting the simple derivation of the globally optimized difference measure from the locally defined ones.