Distributional Gaussian Processes Layers for Out-of-Distribution Detection

TL;DR

This paper introduces a novel hierarchical convolutional Gaussian Process model with Wasserstein-2 space uncertainty propagation, achieving competitive in-distribution segmentation and superior out-of-distribution detection in medical imaging.

Contribution

It proposes a parameter-efficient Bayesian layer for Gaussian Processes using Wasserstein-2 space, enhancing uncertainty propagation and out-of-distribution detection in medical imaging.

Findings

Approaches the performance of deterministic U-Net in segmentation tasks.

Outperforms previous Bayesian networks in out-of-distribution detection.

Code is publicly available for future research.

Abstract

Machine learning models deployed on medical imaging tasks must be equipped with out-of-distribution detection capabilities in order to avoid erroneous predictions. It is unsure whether out-of-distribution detection models reliant on deep neural networks are suitable for detecting domain shifts in medical imaging. Gaussian Processes can reliably separate in-distribution data points from out-of-distribution data points via their mathematical construction. Hence, we propose a parameter efficient Bayesian layer for hierarchical convolutional Gaussian Processes that incorporates Gaussian Processes operating in Wasserstein-2 space to reliably propagate uncertainty. This directly replaces convolving Gaussian Processes with a distance-preserving affine operator on distributions. Our experiments on brain tissue-segmentation show that the resulting architecture approaches the performance of well-established deterministic segmentation algorithms (U-Net), which has not been achieved with previous hierarchical Gaussian Processes. Moreover, by applying the same segmentation model to out-of-distribution data (i.e., images with pathology such as brain tumors), we show that our uncertainty estimates result in out-of-distribution detection that outperforms the capabilities of previous Bayesian networks and reconstruction-based approaches that learn normative distributions. To facilitate future work our code is publicly available.