HyperText: Endowing FastText with Hyperbolic Geometry

TL;DR

This paper introduces HyperText, a hyperbolic geometry-based extension of FastText, which better models hierarchical language structures and improves classification performance with fewer parameters.

Contribution

It presents a novel hyperbolic extension to FastText that enhances hierarchical modeling and efficiency in text classification.

Findings

HyperText outperforms FastText on multiple text classification tasks.

HyperText uses fewer parameters while maintaining or improving accuracy.

Hyperbolic geometry effectively captures hierarchical language structures.

Abstract

Natural language data exhibit tree-like hierarchical structures such as the hypernym-hyponym relations in WordNet. FastText, as the state-of-the-art text classifier based on shallow neural network in Euclidean space, may not model such hierarchies precisely with limited representation capacity. Considering that hyperbolic space is naturally suitable for modeling tree-like hierarchical data, we propose a new model named HyperText for efficient text classification by endowing FastText with hyperbolic geometry. Empirically, we show that HyperText outperforms FastText on a range of text classification tasks with much reduced parameters.