Hierarchical Novelty Detection for Visual Object Recognition

TL;DR

This paper introduces hierarchical novelty detection methods for visual object recognition, enabling models to identify and classify unseen objects by locating their closest known superclass within a taxonomy.

Contribution

It proposes novel top-down and flatten approaches using confidence calibration, data relabeling, and leave-one-out strategies to improve detection of novel classes in a hierarchical framework.

Findings

Effective detection of novel classes within a hierarchy.

Improved zero-shot learning performance with hierarchical embeddings.

Combines multiple strategies for robust novelty detection.

Abstract

Deep neural networks have achieved impressive success in large-scale visual object recognition tasks with a predefined set of classes. However, recognizing objects of novel classes unseen during training still remains challenging. The problem of detecting such novel classes has been addressed in the literature, but most prior works have focused on providing simple binary or regressive decisions, e.g., the output would be "known," "novel," or corresponding confidence intervals. In this paper, we study more informative novelty detection schemes based on a hierarchical classification framework. For an object of a novel class, we aim for finding its closest super class in the hierarchical taxonomy of known classes. To this end, we propose two different approaches termed top-down and flatten methods, and their combination as well. The essential ingredients of our methods are confidence-calibrated classifiers, data relabeling, and the leave-one-out strategy for modeling novel classes under the hierarchical taxonomy. Furthermore, our method can generate a hierarchical embedding that leads to improved generalized zero-shot learning performance in combination with other commonly-used semantic embeddings.