Out-of-Distribution Detection Using an Ensemble of Self Supervised Leave-out Classifiers

TL;DR

This paper introduces an ensemble-based OOD detection method using self-supervised leave-out classifiers with a novel margin-based loss, significantly improving detection performance over existing methods.

Contribution

It proposes a new ensemble approach with a margin-based loss and a novel output combination method for improved out-of-distribution detection.

Findings

Outperforms Hendrycks et al. method on benchmarks

Outperforms ODIN on several benchmarks

Effective in distinguishing OOD from ID samples

Abstract

As deep learning methods form a critical part in commercially important applications such as autonomous driving and medical diagnostics, it is important to reliably detect out-of-distribution (OOD) inputs while employing these algorithms. In this work, we propose an OOD detection algorithm which comprises of an ensemble of classifiers. We train each classifier in a self-supervised manner by leaving out a random subset of training data as OOD data and the rest as in-distribution (ID) data. We propose a novel margin-based loss over the softmax output which seeks to maintain at least a margin $m$ between the average entropy of the OOD and in-distribution samples. In conjunction with the standard cross-entropy loss, we minimize the novel loss to train an ensemble of classifiers. We also propose a novel method to combine the outputs of the ensemble of classifiers to obtain OOD detection score and class prediction. Overall, our method convincingly outperforms Hendrycks et al.[7] and the current state-of-the-art ODIN[13] on several OOD detection benchmarks.