A study on the adequacy of common IQA measures for medical images

TL;DR

This study evaluates the effectiveness of common image quality assessment measures on medical images, revealing that traditional measures like PSNR and SSIM are less suitable, while HaarPSI and other measures perform better but still need tailoring for medical imaging.

Contribution

The paper systematically compares standard IQA measures on medical images and highlights the need for specialized metrics tailored to medical imaging properties.

Findings

HaarPSI outperforms other IQA measures in medical images.

PSNR and SSIM show lower correlation with expert ratings.

Natural image IQA measures yield higher correlations than medical images.

Abstract

Image quality assessment (IQA) is standard practice in the development stage of novel machine learning algorithms that operate on images. The most commonly used IQA measures have been developed and tested for natural images, but not in the medical setting. Reported inconsistencies arising in medical images are not surprising, as they have different properties than natural images. In this study, we test the applicability of common IQA measures for medical image data by comparing their assessment to manually rated chest X-ray (5 experts) and photoacoustic image data (2 experts). Moreover, we include supplementary studies on grayscale natural images and accelerated brain MRI data. The results of all experiments show a similar outcome in line with previous findings for medical images: PSNR and SSIM in the default setting are in the lower range of the result list and HaarPSI outperforms the other tested measures in the overall performance. Also among the top performers in our experiments are the full reference measures FSIM, LPIPS and MS-SSIM. Generally, the results on natural images yield considerably higher correlations, suggesting that additional employment of tailored IQA measures for medical imaging algorithms is needed.