A study of why we need to reassess full reference image quality assessment with medical images

TL;DR

This paper highlights the inadequacy of traditional full reference image quality metrics like PSNR and SSIM for medical images, emphasizing the need for specialized assessment methods to improve AI reliability in clinical settings.

Contribution

It provides a comprehensive review of the limitations of PSNR and SSIM in medical imaging and offers guidelines and future research directions for developing better IQA measures.

Findings

PSNR and SSIM are unsuitable for many medical imaging modalities.

Medical images have properties that differ significantly from natural images.

Improved IQA measures are needed for reliable AI applications in medicine.

Abstract

Image quality assessment (IQA) is indispensable in clinical practice to ensure high standards, as well as in the development stage of machine learning algorithms that operate on medical images. The popular full reference (FR) IQA measures PSNR and SSIM are known and tested for working successfully in many natural imaging tasks, but discrepancies in medical scenarios have been reported in the literature, highlighting the gap between development and actual clinical application. Such inconsistencies are not surprising, as medical images have very different properties than natural images, and PSNR and SSIM have neither been targeted nor properly tested for medical images. This may cause unforeseen problems in clinical applications due to wrong judgment of novel methods. This paper provides a structured and comprehensive overview of examples where PSNR and SSIM prove to be unsuitable for the assessment of novel algorithms using different kinds of medical images, including real-world MRI, CT, OCT, X-Ray, digital pathology and photoacoustic imaging data. Therefore, improvement is urgently needed in particular in this era of AI to increase reliability and explainability in machine learning for medical imaging and beyond. Lastly, we will provide ideas for future research as well as suggesting guidelines for the usage of FR-IQA measures applied to medical images.