Three Applications of Conformal Prediction for Rating Breast Density in Mammography

TL;DR

This paper explores three applications of conformal prediction to improve trust, reliability, and fairness in deep learning models for mammographic breast density assessment, addressing key challenges in clinical deployment.

Contribution

It provides a detailed analysis of conformal prediction applications in medical imaging, demonstrating their potential to enhance uncertainty quantification and clinical trust.

Findings

Conformal prediction helps characterize distribution shifts in medical imaging.

It improves prediction quality and confidence in breast density assessment.

Enhances subgroup fairness and reliability of AI models in mammography.

Abstract

Breast cancer is the most common cancers and early detection from mammography screening is crucial in improving patient outcomes. Assessing mammographic breast density is clinically important as the denser breasts have higher risk and are more likely to occlude tumors. Manual assessment by experts is both time-consuming and subject to inter-rater variability. As such, there has been increased interest in the development of deep learning methods for mammographic breast density assessment. Despite deep learning having demonstrated impressive performance in several prediction tasks for applications in mammography, clinical deployment of deep learning systems in still relatively rare; historically, mammography Computer-Aided Diagnoses (CAD) have over-promised and failed to deliver. This is in part due to the inability to intuitively quantify uncertainty of the algorithm for the clinician, which would greatly enhance usability. Conformal prediction is well suited to increase reliably and trust in deep learning tools but they lack realistic evaluations on medical datasets. In this paper, we present a detailed analysis of three possible applications of conformal prediction applied to medical imaging tasks: distribution shift characterization, prediction quality improvement, and subgroup fairness analysis. Our results show the potential of distribution-free uncertainty quantification techniques to enhance trust on AI algorithms and expedite their translation to usage.