Combining Deep Universal Features, Semantic Attributes, and Hierarchical Classification for Zero-Shot Learning

TL;DR

This paper enhances zero-shot learning by integrating deep universal features, semantic attributes, and hierarchical classification, demonstrating improved performance on ImageNet classes for both novel and known categories.

Contribution

It introduces a combined approach that leverages semantic attributes and hierarchical classification to improve zero-shot learning performance.

Findings

Semantic attribute-based posteriors improve novel class recognition.

Different hierarchical methods optimize for non-novel versus novel classes.

Using deep universal features enhances zero-shot learning accuracy.

Abstract

We address zero-shot (ZS) learning, building upon prior work in hierarchical classification by combining it with approaches based on semantic attribute estimation. For both non-novel and novel image classes we compare multiple formulations of the problem, starting with deep universal features in each case. We investigate the effect of using different posterior probabilities as inputs to the hierarchical classifier, comparing the performances of posteriors derived from distances to SVM classifier boundaries with those of posteriors based on semantic attribute estimation. Using a dataset consisting of 150 object classes from the ImageNet ILSVRC2012 data set, we find that the hierarchical classification method that maximizes expected reward for non-novel classes differs from the method that maximizes expected reward for novel classes. We also show that using input posteriors based on semantic attributes improves the expected reward for novel classes.