Uncertainty-based quality assurance of carotid artery wall segmentation in black-blood MRI

TL;DR

This study explores using model uncertainty estimates to automatically assess the quality of carotid artery wall segmentations in black-blood MRI, facilitating large-scale data analysis.

Contribution

It demonstrates that uncertainty metrics can effectively serve as proxies for segmentation quality and detect low-quality results without degrading segmentation performance.

Findings

Uncertainty measures correlate with segmentation accuracy.

Uncertainty metrics can identify low-quality segmentations.

Including uncertainty does not impair segmentation quality.

Abstract

The application of deep learning models to large-scale data sets requires means for automatic quality assurance. We have previously developed a fully automatic algorithm for carotid artery wall segmentation in black-blood MRI that we aim to apply to large-scale data sets. This method identifies nested artery walls in 3D patches centered on the carotid artery. In this study, we investigate to what extent the uncertainty in the model predictions for the contour location can serve as a surrogate for error detection and, consequently, automatic quality assurance. We express the quality of automatic segmentations using the Dice similarity coefficient. The uncertainty in the model's prediction is estimated using either Monte Carlo dropout or test-time data augmentation. We found that (1) including uncertainty measurements did not degrade the quality of the segmentations, (2) uncertainty metrics provide a good proxy of the quality of our contours if the center found during the first step is enclosed in the lumen of the carotid artery and (3) they could be used to detect low-quality segmentations at the participant level. This automatic quality assurance tool might enable the application of our model in large-scale data sets.