# Assessing image quality in photoacoustic imaging: A metric-based and deep learning-based evaluation

## TL;DR

This paper benchmarks image quality assessment methods for photoacoustic imaging, showing that structure-based metrics and deep learning models can improve automated quality evaluation.

## Contribution

The first large-scale benchmark of IQA in PA imaging, evaluating 13 metrics and deep learning models across nearly one million images.

## Key findings

- Structural similarity-based metrics like SSIM reliably capture image quality differences in PAI.
- Deep learning models, especially PAQNet, achieve high correlations with reference-based scores for no-reference quality estimation.
- Conventional metrics like PSNR and existing NR metrics correlate poorly with reconstruction improvements.

## Abstract

Photoacoustic (PA) imaging offers high-resolution functional imaging in vivo, but image quality varies with acquisition hardware, reconstruction methods, and scanning conditions. Reliable image quality assessment (IQA) is therefore critical for evaluating new PA technologies and ensuring reproducible research. IQA metrics originally proposed for natural images are widely used; however, their relevance to PA imaging has not been systematically benchmarked. We present the first large-scale benchmark of IQA in PA imaging, evaluating 11 full-reference (FR) metrics and 2 no-reference (NR) metrics across nearly one million PA images from five independent datasets. These datasets span phantoms, preclinical small-animal scans, and challenging ex vivo and in vivo acquisitions with controlled degradations across multiple commercial imaging systems. Metric scores were analyzed statistically to determine which metrics best distinguish differences in image quality. Metrics that showed consistent performance across datasets are modeled using three deep learning architectures (PAQNet, IQDCNN, and EfficientNetIQA), where we trained the models to predict these metric values directly from PA images, enabling automated no-reference quality estimation. The results show that structural similarity-based FR metrics, especially Structural Similarity Index Measure (SSIM) and its variants, reliably capture image quality differences in PAI, whereas conventional metrics like Peak Signal-to-Noise Ratio (PSNR) and existing NR metrics correlate poorly with reconstruction improvements. Deep learning models, particularly PAQNet, achieved high correlations with reference-based scores and provide a practical path to reference-free quality assessment, although generalization across datasets from different systems remains challenging. This study establishes a benchmark for PA image quality evaluation and demonstrates that structure-aware metrics and learned no-reference predictors can enable more reliable and automated quality assessment in PAI. We provide code for IQA metrics evaluation and deep learning models for reproducibility and further development at https://github.com/MellevdB/photoacoustic-image-quality-assessment.git.