Plugin estimators for selective classification with out-of-distribution detection

TL;DR

This paper introduces theoretically grounded plugin estimators for selective classification with out-of-distribution detection, unifying and improving upon existing heuristic methods in the field.

Contribution

It proposes new plugin estimators for SCOD that are theoretically justified, effective, and generalize existing approaches from SC and OOD detection.

Findings

The proposed estimators outperform heuristic baselines in experiments.

Naive use of existing SC and OOD methods may be inadequate for SCOD.

The approach provides a unified framework for SC and OOD detection.

Abstract

Real-world classifiers can benefit from the option of abstaining from predicting on samples where they have low confidence. Such abstention is particularly useful on samples which are close to the learned decision boundary, or which are outliers with respect to the training sample. These settings have been the subject of extensive but disjoint study in the selective classification (SC) and out-of-distribution (OOD) detection literature. Recent work on selective classification with OOD detection (SCOD) has argued for the unified study of these problems; however, the formal underpinnings of this problem are still nascent, and existing techniques are heuristic in nature. In this paper, we propose new plugin estimators for SCOD that are theoretically grounded, effective, and generalise existing approaches from the SC and OOD detection literature. In the course of our analysis, we formally explicate how na\"{i}ve use of existing SC and OOD detection baselines may be inadequate for SCOD. We empirically demonstrate that our approaches yields competitive SC and OOD detection performance compared to baselines from both literatures.