Detecting Novelties with Empty Classes

TL;DR

This paper presents a method for enabling deep neural networks to detect and learn novel classes in an unsupervised manner by extending their output space with empty classes and fine-tuning on out-of-distribution data.

Contribution

It introduces a novel approach using empty classes and specialized loss functions to allow DNNs to identify and incorporate unseen classes without ground truth labels.

Findings

Effective detection of out-of-distribution data

Successful extension of semantic space with new classes

Applicable to image classification and segmentation

Abstract

For open world applications, deep neural networks (DNNs) need to be aware of previously unseen data and adaptable to evolving environments. Furthermore, it is desirable to detect and learn novel classes which are not included in the DNNs underlying set of semantic classes in an unsupervised fashion. The method proposed in this article builds upon anomaly detection to retrieve out-of-distribution (OoD) data as candidates for new classes. We thereafter extend the DNN by $k$ empty classes and fine-tune it on the OoD data samples. To this end, we introduce two loss functions, which 1) entice the DNN to assign OoD samples to the empty classes and 2) to minimize the inner-class feature distances between them. Thus, instead of ground truth which contains labels for the different novel classes, the DNN obtains a single OoD label together with a distance matrix, which is computed in advance. We perform several experiments for image classification and semantic segmentation, which demonstrate that a DNN can extend its own semantic space by multiple classes without having access to ground truth.