Joint Segmentation and Image Reconstruction with Error Prediction in Photoacoustic Imaging using Deep Learning

TL;DR

This paper introduces a hybrid Bayesian CNN that jointly reconstructs photoacoustic images and segmentations while predicting errors, enabling uncertainty quantification and improved validation in PA imaging.

Contribution

The paper presents a novel hybrid Bayesian CNN that jointly predicts PA images, segmentations, and error estimates, enhancing validation and confidence in PA image reconstruction.

Findings

Accurate PA images and segmentations achieved.

Error predictions strongly correlate with actual errors.

Confidence processing improves image reliability.

Abstract

Deep learning has been used to improve photoacoustic (PA) image reconstruction. One major challenge is that errors cannot be quantified to validate predictions when ground truth is unknown. Validation is key to quantitative applications, especially using limited-bandwidth ultrasonic linear detector arrays. Here, we propose a hybrid Bayesian convolutional neural network (Hybrid-BCNN) to jointly predict PA image and segmentation with error (uncertainty) predictions. Each output pixel represents a probability distribution where error can be quantified. The Hybrid-BCNN was trained with simulated PA data and applied to both simulations and experiments. Due to the sparsity of PA images, segmentation focuses Hybrid-BCNN on minimizing the loss function in regions with PA signals for better predictions. The results show that accurate PA segmentations and images are obtained, and error predictions are highly statistically correlated to actual errors. To leverage error predictions, confidence processing created PA images above a specific confidence level.