Structural Similarity Index SSIMplified: Is there really a simpler concept at the heart of image quality measurement?

TL;DR

This paper reinterprets the Structural Similarity Index (SSIM) as a perceptual noise visibility function, simplifying its conceptual foundation and proposing a more straightforward alternative metric for image quality assessment.

Contribution

It reveals SSIM's core as a normalized error or noise visibility function, simplifying its understanding and suggesting a more transparent, less assumption-dependent metric called Dissimilarity Quotient (DQ).

Findings

SSIM is essentially a contrast or visibility function.

The covariance in SSIM equals the difference of variances.

NVF/DQ explains SSIM's success in modeling perceptual quality.

Abstract

The Structural Similarity Index (SSIM) is generally considered to be a milestone in the recent history of Image Quality Assessment (IQA). Alas, SSIM's accepted development from the product of three heuristic factors continues to obscure it's real underlying simplicity. Starting instead from a symmetric-antisymmetric reformulation we first show SSIM to be a contrast or visibility function in the classic sense. Furthermore, the previously enigmatic structural covariance is revealed to be the difference of variances. The second step, eliminating the intrinsic quadratic nature of SSIM, allows a near linear correlation with human observer scores, and without invoking the usual, but arbitrary, sigmoid model fitting. We conclude that SSIM can be re-interpreted in terms of perceptual masking: it is essentially equivalent to a normalised error or noise visibility function (NVF), and, furthermore, the NVF alone explains it success in modelling perceptual image quality. We use the term Dissimilarity Quotient (DQ) for the specifically anti/symmetric SSIM derived NVF. It seems that IQA researchers may now have two choices: 1) Continue to use the complex SSIM formula, but noting that SSIM only works coincidentally since the covariance term is actually the mean square error (MSE) in disguise. 2) Use the simplest of all perceptually-masked image quality metrics, namely NVF or DQ. On this choice Occam is clear: in the absence of differences in predictive ability, the fewer assumptions that are made, the better.