Making Better Mistakes: Leveraging Class Hierarchies with Deep Networks

TL;DR

This paper enhances deep neural network image classification by incorporating class hierarchies into the loss function, reducing severe mistakes and improving performance on complex datasets.

Contribution

It proposes two simple modifications to the cross-entropy loss that leverage class hierarchies, outperforming previous methods on large, complex datasets.

Findings

Improved accuracy on tieredImageNet and iNaturalist'19 datasets.

Reduced severity of classification mistakes.

Outperformed prior hierarchy-based approaches.

Abstract

Deep neural networks have improved image classification dramatically over the past decade, but have done so by focusing on performance measures that treat all classes other than the ground truth as equally wrong. This has led to a situation in which mistakes are less likely to be made than before, but are equally likely to be absurd or catastrophic when they do occur. Past works have recognised and tried to address this issue of mistake severity, often by using graph distances in class hierarchies, but this has largely been neglected since the advent of the current deep learning era in computer vision. In this paper, we aim to renew interest in this problem by reviewing past approaches and proposing two simple modifications of the cross-entropy loss which outperform the prior art under several metrics on two large datasets with complex class hierarchies: tieredImageNet and iNaturalist'19.