Word Sense Disambiguation with LSTM: Do We Really Need 100 Billion Words?

TL;DR

This study reproduces a previous LSTM-based WSD approach using open datasets and software, demonstrating comparable results with significantly less data than originally claimed.

Contribution

It provides a reproducibility analysis of LSTM-based WSD, showing that high performance does not require extremely large datasets.

Findings

State-of-the-art results achieved with less data

Open-source code and models released for community use

Reproduction confirms effectiveness of LSTM for WSD

Abstract

Recently, Yuan et al. (2016) have shown the effectiveness of using Long Short-Term Memory (LSTM) for performing Word Sense Disambiguation (WSD). Their proposed technique outperformed the previous state-of-the-art with several benchmarks, but neither the training data nor the source code was released. This paper presents the results of a reproduction study of this technique using only openly available datasets (GigaWord, SemCore, OMSTI) and software (TensorFlow). From them, it emerged that state-of-the-art results can be obtained with much less data than hinted by Yuan et al. All code and trained models are made freely available.