Harnessing Superclasses for Learning from Hierarchical Databases

TL;DR

This paper proposes a hierarchical loss function for classification tasks that leverages class hierarchies to improve accuracy and reduce coarse errors without additional computational cost.

Contribution

It introduces a proper scoring loss that incorporates class hierarchies, enabling consistent classification across different levels of granularity.

Findings

Improves accuracy across benchmarks

Reduces coarse classification errors

No significant computational overhead

Abstract

In many large-scale classification problems, classes are organized in a known hierarchy, typically represented as a tree expressing the inclusion of classes in superclasses. We introduce a loss for this type of supervised hierarchical classification. It utilizes the knowledge of the hierarchy to assign each example not only to a class but also to all encompassing superclasses. Applicable to any feedforward architecture with a softmax output layer, this loss is a proper scoring rule, in that its expectation is minimized by the true posterior class probabilities. This property allows us to simultaneously pursue consistent classification objectives between superclasses and fine-grained classes, and eliminates the need for a performance trade-off between different granularities. We conduct an experimental study on three reference benchmarks, in which we vary the size of the training sets to cover a diverse set of learning scenarios. Our approach does not entail any significant additional computational cost compared with the loss of cross-entropy. It improves accuracy and reduces the number of coarse errors, with predicted labels that are distant from ground-truth labels in the tree.