Multi-layer Radial Basis Function Networks for Out-of-distribution Detection

TL;DR

This paper introduces a multi-layer radial basis function network (MLRBFN) that simplifies out-of-distribution detection by integrating classification and OOD detection into a single, trainable architecture, showing competitive results.

Contribution

The paper proposes a novel, easily trainable multi-layer RBFN architecture with a depression mechanism for effective OOD detection, unifying classification and OOD detection.

Findings

MLRBFNs are competitive with existing OOD detection methods.

The architecture can be used as standalone classifiers or with pretrained features.

The depression mechanism enhances OOD detection performance.

Abstract

Existing methods for out-of-distribution (OOD) detection use various techniques to produce a score, separate from classification, that determines how ``OOD'' an input is. Our insight is that OOD detection can be simplified by using a neural network architecture which can effectively merge classification and OOD detection into a single step. Radial basis function networks (RBFNs) inherently link classification confidence and OOD detection; however, these networks have lost popularity due to the difficult of training them in a multi-layer fashion. In this work, we develop a multi-layer radial basis function network (MLRBFN) which can be easily trained. To ensure that these networks are also effective for OOD detection, we develop a novel depression mechanism. We apply MLRBFNs as standalone classifiers and as heads on top of pretrained feature extractors, and find that they are competitive with commonly used methods for OOD detection. Our MLRBFN architecture demonstrates a promising new direction for OOD detection methods.