An Analysis of Hierarchical Text Classification Using Word Embeddings

TL;DR

This paper evaluates the effectiveness of word embeddings combined with machine learning algorithms for hierarchical text classification, demonstrating promising results on standard datasets.

Contribution

It systematically assesses various word embeddings and classifiers for HTC, providing empirical evidence of their effectiveness.

Findings

FastText achieved an ${}_{LCA}F_1$ of 0.893 on RCV1.

Word embeddings significantly improve HTC performance.

Analysis confirms the promise of embedding-based methods for hierarchical classification.

Abstract

Efficient distributed numerical word representation models (word embeddings) combined with modern machine learning algorithms have recently yielded considerable improvement on automatic document classification tasks. However, the effectiveness of such techniques has not been assessed for the hierarchical text classification (HTC) yet. This study investigates the application of those models and algorithms on this specific problem by means of experimentation and analysis. We trained classification models with prominent machine learning algorithm implementations---fastText, XGBoost, SVM, and Keras' CNN---and noticeable word embeddings generation methods---GloVe, word2vec, and fastText---with publicly available data and evaluated them with measures specifically appropriate for the hierarchical context. FastText achieved an ${}_{LCA}F_1$ of 0.893 on a single-labeled version of the RCV1 dataset. An analysis indicates that using word embeddings and its flavors is a very promising approach for HTC.