HardBoost: Boosting Zero-Shot Learning with Hard Classes

TL;DR

This paper systematically analyzes the hard class problem in zero-shot learning, identifies high semantic affinity as a cause, and proposes frameworks to detect and exploit hard classes, significantly improving ZSL performance.

Contribution

It introduces metrics for detecting hard classes and presents frameworks that enhance existing ZSL methods by leveraging hard class information.

Findings

Hard class problem is widespread across ZSL methods.

High semantic affinity among unseen classes contributes to hardness.

Exploiting hard classes improves ZSL performance significantly.

Abstract

This work is a systematical analysis on the so-called hard class problem in zero-shot learning (ZSL), that is, some unseen classes disproportionally affect the ZSL performances than others, as well as how to remedy the problem by detecting and exploiting hard classes. At first, we report our empirical finding that the hard class problem is a ubiquitous phenomenon and persists regardless of used specific methods in ZSL. Then, we find that high semantic affinity among unseen classes is a plausible underlying cause of hardness and design two metrics to detect hard classes. Finally, two frameworks are proposed to remedy the problem by detecting and exploiting hard classes, one under inductive setting, the other under transductive setting. The proposed frameworks could accommodate most existing ZSL methods to further significantly boost their performances with little efforts. Extensive experiments on three popular benchmarks demonstrate the benefits by identifying and exploiting the hard classes in ZSL.