PAC Bayesian Performance Guarantees for Deep (Stochastic) Networks in Medical Imaging

TL;DR

This paper applies PAC-Bayesian theoretical bounds to deep stochastic networks in medical imaging, providing generalization guarantees that help address overfitting concerns in small datasets.

Contribution

It extends PAC-Bayesian bounds to medical imaging tasks, including classification and segmentation, demonstrating their effectiveness and interpretability on a small dataset.

Findings

Bounds are competitive with simpler baselines.

Bounds are more explainable than traditional methods.

Eliminates the need for holdout sets in evaluation.

Abstract

Application of deep neural networks to medical imaging tasks has in some sense become commonplace. Still, a "thorn in the side" of the deep learning movement is the argument that deep networks are prone to overfitting and are thus unable to generalize well when datasets are small (as is common in medical imaging tasks). One way to bolster confidence is to provide mathematical guarantees, or bounds, on network performance after training which explicitly quantify the possibility of overfitting. In this work, we explore recent advances using the PAC-Bayesian framework to provide bounds on generalization error for large (stochastic) networks. While previous efforts focus on classification in larger natural image datasets (e.g., MNIST and CIFAR-10), we apply these techniques to both classification and segmentation in a smaller medical imagining dataset: the ISIC 2018 challenge set. We observe the resultant bounds are competitive compared to a simpler baseline, while also being more explainable and alleviating the need for holdout sets.