Reducing Network Agnostophobia

TL;DR

This paper introduces novel loss functions, Entropic Open-Set and Objectosphere, to improve deep networks' ability to reject unknown inputs, demonstrated through experiments on MNIST and CIFAR-10 datasets.

Contribution

The paper proposes simple yet effective loss functions that enhance neural networks' open-set recognition by increasing entropy and feature separation for unknown inputs.

Findings

Significantly better handling of unknown inputs with the new losses.

Effective in classifying known classes while rejecting unknowns.

Validated on multiple datasets including Devanagari, NotMNIST, CIFAR-100, and SVHN.

Abstract

Agnostophobia, the fear of the unknown, can be experienced by deep learning engineers while applying their networks to real-world applications. Unfortunately, network behavior is not well defined for inputs far from a networks training set. In an uncontrolled environment, networks face many instances that are not of interest to them and have to be rejected in order to avoid a false positive. This problem has previously been tackled by researchers by either a) thresholding softmax, which by construction cannot return "none of the known classes", or b) using an additional background or garbage class. In this paper, we show that both of these approaches help, but are generally insufficient when previously unseen classes are encountered. We also introduce a new evaluation metric that focuses on comparing the performance of multiple approaches in scenarios where such unseen classes or unknowns are encountered. Our major contributions are simple yet effective Entropic Open-Set and Objectosphere losses that train networks using negative samples from some classes. These novel losses are designed to maximize entropy for unknown inputs while increasing separation in deep feature space by modifying magnitudes of known and unknown samples. Experiments on networks trained to classify classes from MNIST and CIFAR-10 show that our novel loss functions are significantly better at dealing with unknown inputs from datasets such as Devanagari, NotMNIST, CIFAR-100, and SVHN.