TaxoKnow: Taxonomy as Prior Knowledge in the Loss Function of Multi-class Classification

TL;DR

This paper explores integrating hierarchical taxonomies into neural network loss functions to improve multi-class classification, demonstrating significant performance gains in semi-supervised and supervised settings.

Contribution

It introduces two methods to incorporate taxonomy as a regularizer in the loss function, enhancing classification performance by leveraging hierarchical label information.

Findings

Taxonomy integration improves semi-supervised classification accuracy.

Significant performance gains in fully supervised classification.

Effective on both industrial and public benchmark datasets.

Abstract

In this paper, we investigate the effectiveness of integrating a hierarchical taxonomy of labels as prior knowledge into the learning algorithm of a flat classifier. We introduce two methods to integrate the hierarchical taxonomy as an explicit regularizer into the loss function of learning algorithms. By reasoning on a hierarchical taxonomy, a neural network alleviates its output distributions over the classes, allowing conditioning on upper concepts for a minority class. We limit ourselves to the flat classification task and provide our experimental results on two industrial in-house datasets and two public benchmarks, RCV1 and Amazon product reviews. Our obtained results show the significant effect of a taxonomy in increasing the performance of a learner in semisupervised multi-class classification and the considerable results obtained in a fully supervised fashion.