The Extreme Value Machine

TL;DR

The paper introduces the Extreme Value Machine (EVM), a theoretically grounded classifier capable of incremental learning and recognizing unseen classes, with demonstrated accuracy and efficiency on ImageNet features.

Contribution

It presents the EVM, a novel EVT-based classifier that enables nonlinear, kernel-free, incremental learning for recognition tasks involving unseen classes.

Findings

EVM is accurate and efficient on ImageNet features.

EVM can perform incremental learning with unknown classes.

First EVT-based classifier for nonlinear, kernel-free incremental learning.

Abstract

It is often desirable to be able to recognize when inputs to a recognition function learned in a supervised manner correspond to classes unseen at training time. With this ability, new class labels could be assigned to these inputs by a human operator, allowing them to be incorporated into the recognition function --- ideally under an efficient incremental update mechanism. While good algorithms that assume inputs from a fixed set of classes exist, e.g., artificial neural networks and kernel machines, it is not immediately obvious how to extend them to perform incremental learning in the presence of unknown query classes. Existing algorithms take little to no distributional information into account when learning recognition functions and lack a strong theoretical foundation. We address this gap by formulating a novel, theoretically sound classifier --- the Extreme Value Machine (EVM). The EVM has a well-grounded interpretation derived from statistical Extreme Value Theory (EVT), and is the first classifier to be able to perform nonlinear kernel-free variable bandwidth incremental learning. Compared to other classifiers in the same deep network derived feature space, the EVM is accurate and efficient on an established benchmark partition of the ImageNet dataset.