Selective Zero-Shot Classification with Augmented Attributes

TL;DR

This paper addresses the challenge of selective zero-shot classification by combining human-defined attributes with automatically discovered residual attributes, improving prediction confidence and performance in avoiding dubious predictions.

Contribution

It introduces a novel selective zero-shot classifier that jointly learns and utilizes both human-defined and residual attributes for better decision-making.

Findings

Outperforms existing methods on multiple benchmarks.

Improves risk-coverage trade-off in zero-shot classification.

Effectively identifies and avoids dubious predictions.

Abstract

In this paper, we introduce a selective zero-shot classification problem: how can the classifier avoid making dubious predictions? Existing attribute-based zero-shot classification methods are shown to work poorly in the selective classification scenario. We argue the under-complete human defined attribute vocabulary accounts for the poor performance. We propose a selective zero-shot classifier based on both the human defined and the automatically discovered residual attributes. The proposed classifier is constructed by firstly learning the defined and the residual attributes jointly. Then the predictions are conducted within the subspace of the defined attributes. Finally, the prediction confidence is measured by both the defined and the residual attributes. Experiments conducted on several benchmarks demonstrate that our classifier produces a superior performance to other methods under the risk-coverage trade-off metric.