Batch-Ensemble Stochastic Neural Networks for Out-of-Distribution Detection

TL;DR

This paper introduces Batch-Ensemble Stochastic Neural Networks (BE-SNNs), a novel method for out-of-distribution detection that combines uncertainty quantification with an efficient ensemble mechanism, demonstrating superior performance on multiple benchmarks.

Contribution

The paper proposes BE-SNNs, integrating feature distribution modeling with batch-ensemble to improve OOD detection and address feature collapse.

Findings

BE-SNNs outperform state-of-the-art methods on various OOD benchmarks.

The approach effectively models feature uncertainty for better detection.

BE-SNNs are computationally efficient due to the batch-ensemble mechanism.

Abstract

Out-of-distribution (OOD) detection has recently received much attention from the machine learning community due to its importance in deploying machine learning models in real-world applications. In this paper we propose an uncertainty quantification approach by modelling the distribution of features. We further incorporate an efficient ensemble mechanism, namely batch-ensemble, to construct the batch-ensemble stochastic neural networks (BE-SNNs) and overcome the feature collapse problem. We compare the performance of the proposed BE-SNNs with the other state-of-the-art approaches and show that BE-SNNs yield superior performance on several OOD benchmarks, such as the Two-Moons dataset, the FashionMNIST vs MNIST dataset, FashionMNIST vs NotMNIST dataset, and the CIFAR10 vs SVHN dataset.