ROOT13: Spotting Hypernyms, Co-Hyponyms and Randoms

TL;DR

ROOT13 is a supervised system that accurately classifies hypernyms, co-hyponyms, and random word pairs using a Random Forest with corpus-based features, outperforming baseline methods.

Contribution

The paper introduces ROOT13, a novel supervised classification system utilizing 13 features and Random Forests for hypernymy and co-hyponymy detection, achieving high accuracy.

Findings

F1 score of 88.3% on 9,600 pairs with all classes present

High binary classification accuracy: hypernyms-co-hyponyms 93.4%

Outperforms baseline cosine similarity approach

Abstract

In this paper, we describe ROOT13, a supervised system for the classification of hypernyms, co-hyponyms and random words. The system relies on a Random Forest algorithm and 13 unsupervised corpus-based features. We evaluate it with a 10-fold cross validation on 9,600 pairs, equally distributed among the three classes and involving several Parts-Of-Speech (i.e. adjectives, nouns and verbs). When all the classes are present, ROOT13 achieves an F1 score of 88.3%, against a baseline of 57.6% (vector cosine). When the classification is binary, ROOT13 achieves the following results: hypernyms-co-hyponyms (93.4% vs. 60.2%), hypernymsrandom (92.3% vs. 65.5%) and co-hyponyms-random (97.3% vs. 81.5%). Our results are competitive with stateof-the-art models.