WSD algorithm based on a new method of vector-word contexts proximity calculation via epsilon-filtration

TL;DR

This paper introduces a novel WSD algorithm using epsilon-filtration for vector-word context proximity, demonstrating improved accuracy in disambiguating words in sentences with a tagged corpus.

Contribution

The paper proposes a new WSD algorithm based on epsilon-filtering of word contexts, enhancing disambiguation accuracy and providing an open-source implementation and corpus.

Findings

The new algorithm outperforms some existing methods in certain cases.

Extensive experiments validate the effectiveness of the epsilon-filtration approach.

Open access to software and tagged corpus supports further research.

Abstract

The problem of word sense disambiguation (WSD) is considered in the article. Given a set of synonyms (synsets) and sentences with these synonyms. It is necessary to select the meaning of the word in the sentence automatically. 1285 sentences were tagged by experts, namely, one of the dictionary meanings was selected by experts for target words. To solve the WSD-problem, an algorithm based on a new method of vector-word contexts proximity calculation is proposed. In order to achieve higher accuracy, a preliminary epsilon-filtering of words is performed, both in the sentence and in the set of synonyms. An extensive program of experiments was carried out. Four algorithms are implemented, including a new algorithm. Experiments have shown that in a number of cases the new algorithm shows better results. The developed software and the tagged corpus have an open license and are available online. Wiktionary and Wikisource are used. A brief description of this work can be viewed in slides (https://goo.gl/9ak6Gt). Video lecture in Russian on this research is available online (https://youtu.be/-DLmRkepf58).