Global Hierarchical Neural Networks using Hierarchical Softmax

TL;DR

This paper introduces a hierarchical softmax framework for global hierarchical classification, demonstrating improved performance over flat softmax classifiers across multiple text datasets.

Contribution

It proposes a novel hierarchical softmax-based classifier applicable to any task with a class hierarchy, showing empirical improvements over traditional softmax.

Findings

Hierarchical softmax outperforms flat softmax in macro-F1 and macro-recall.

In three datasets, it also improves micro-accuracy and macro-precision.

The approach is effective across diverse text classification datasets.

Abstract

This paper presents a framework in which hierarchical softmax is used to create a global hierarchical classifier. The approach is applicable for any classification task where there is a natural hierarchy among classes. We show empirical results on four text classification datasets. In all datasets the hierarchical softmax improved on the regular softmax used in a flat classifier in terms of macro-F1 and macro-recall. In three out of four datasets hierarchical softmax achieved a higher micro-accuracy and macro-precision.