Word embeddings and recurrent neural networks based on Long-Short Term Memory nodes in supervised biomedical word sense disambiguation

TL;DR

This paper demonstrates that combining word embeddings with traditional features and LSTM-based neural networks significantly improves supervised biomedical word sense disambiguation accuracy.

Contribution

It introduces the use of word embeddings and LSTM neural networks for biomedical word sense disambiguation, achieving state-of-the-art results.

Findings

Word embeddings enhance traditional feature performance.

LSTM classifiers outperform other models.

Achieved 95.97% macro accuracy on MSH WSD dataset.

Abstract

Word sense disambiguation helps identifying the proper sense of ambiguous words in text. With large terminologies such as the UMLS Metathesaurus ambiguities appear and highly effective disambiguation methods are required. Supervised learning algorithm methods are used as one of the approaches to perform disambiguation. Features extracted from the context of an ambiguous word are used to identify the proper sense of such a word. The type of features have an impact on machine learning methods, thus affect disambiguation performance. In this work, we have evaluated several types of features derived from the context of the ambiguous word and we have explored as well more global features derived from MEDLINE using word embeddings. Results show that word embeddings improve the performance of more traditional features and allow as well using recurrent neural network classifiers based on Long-Short Term Memory (LSTM) nodes. The combination of unigrams and word embeddings with an SVM sets a new state of the art performance with a macro accuracy of 95.97 in the MSH WSD data set.