Sparse Bayesian Networks: Efficient Uncertainty Quantification in Medical Image Analysis

TL;DR

This paper proposes a sparse Bayesian network training method that selectively makes parameters Bayesian, drastically reducing computational costs while maintaining high performance and uncertainty quantification in medical image analysis.

Contribution

It introduces a novel training procedure for sparse Bayesian networks that significantly reduces Bayesian parameters and computational costs without sacrificing accuracy.

Findings

Reduces Bayesian parameters by over 95%.

Achieves competitive performance in classification and segmentation tasks.

Significantly lowers training and inference costs compared to full Bayesian methods.

Abstract

Efficiently quantifying predictive uncertainty in medical images remains a challenge. While Bayesian neural networks (BNN) offer predictive uncertainty, they require substantial computational resources to train. Although Bayesian approximations such as ensembles have shown promise, they still suffer from high training and inference costs. Existing approaches mainly address the costs of BNN inference post-training, with little focus on improving training efficiency and reducing parameter complexity. This study introduces a training procedure for a sparse (partial) Bayesian network. Our method selectively assigns a subset of parameters as Bayesian by assessing their deterministic saliency through gradient sensitivity analysis. The resulting network combines deterministic and Bayesian parameters, exploiting the advantages of both representations to achieve high task-specific performance and minimize predictive uncertainty. Demonstrated on multi-label ChestMNIST for classification and ISIC, LIDC-IDRI for segmentation, our approach achieves competitive performance and predictive uncertainty estimation by reducing Bayesian parameters by over 95\%, significantly reducing computational expenses compared to fully Bayesian and ensemble methods.