A Dempster-Shafer approach to trustworthy AI with application to fetal brain MRI segmentation

TL;DR

This paper introduces a Dempster-Shafer based framework to enhance the trustworthiness of AI in fetal brain MRI segmentation, automatically discarding unreliable predictions and improving robustness across diverse datasets.

Contribution

It proposes a novel trustworthy AI framework that integrates fallback mechanisms using Dempster-Shafer theory to detect and correct model failures in medical image segmentation.

Findings

Improves robustness of fetal brain MRI segmentation across multiple centers.

Effectively discards predictions violating expert knowledge.

Enhances safety and reliability of AI in clinical settings.

Abstract

Deep learning models for medical image segmentation can fail unexpectedly and spectacularly for pathological cases and images acquired at different centers than training images, with labeling errors that violate expert knowledge. Such errors undermine the trustworthiness of deep learning models for medical image segmentation. Mechanisms for detecting and correcting such failures are essential for safely translating this technology into clinics and are likely to be a requirement of future regulations on artificial intelligence (AI). In this work, we propose a trustworthy AI theoretical framework and a practical system that can augment any backbone AI system using a fallback method and a fail-safe mechanism based on Dempster-Shafer theory. Our approach relies on an actionable definition of trustworthy AI. Our method automatically discards the voxel-level labeling predicted by the backbone AI that violate expert knowledge and relies on a fallback for those voxels. We demonstrate the effectiveness of the proposed trustworthy AI approach on the largest reported annotated dataset of fetal MRI consisting of 540 manually annotated fetal brain 3D T2w MRIs from 13 centers. Our trustworthy AI method improves the robustness of a state-of-the-art backbone AI for fetal brain MRIs acquired across various centers and for fetuses with various brain abnormalities.