Selective Classification for Deep Neural Networks

TL;DR

This paper introduces a method for selective classification in deep neural networks that allows controlling the risk level by rejecting uncertain instances, improving reliability in critical applications.

Contribution

It presents a novel approach to construct selective classifiers from trained DNNs, enabling risk control and high-confidence predictions with theoretical guarantees.

Findings

Achieves 2% error rate in top-5 ImageNet classification with 99.9% probability

Enables high-confidence predictions with nearly 60% coverage on ImageNet

Demonstrates effectiveness on CIFAR and ImageNet datasets

Abstract

Selective classification techniques (also known as reject option) have not yet been considered in the context of deep neural networks (DNNs). These techniques can potentially significantly improve DNNs prediction performance by trading-off coverage. In this paper we propose a method to construct a selective classifier given a trained neural network. Our method allows a user to set a desired risk level. At test time, the classifier rejects instances as needed, to grant the desired risk (with high probability). Empirical results over CIFAR and ImageNet convincingly demonstrate the viability of our method, which opens up possibilities to operate DNNs in mission-critical applications. For example, using our method an unprecedented 2% error in top-5 ImageNet classification can be guaranteed with probability 99.9%, and almost 60% test coverage.