SenseFitting: Sense Level Semantic Specialization of Word Embeddings for Word Sense Disambiguation

TL;DR

SenseFitting is a novel neural network approach for German word sense disambiguation that leverages sense-level semantic specialization to outperform existing methods and establish new state-of-the-art results.

Contribution

It introduces SenseFitting, a new method for optimizing sense embeddings using lexical-semantic constraints, improving German WSD performance.

Findings

Outperforms knowledge-based WSD methods by up to 25% F1-score

Achieves state-of-the-art results on WebCAGe dataset

Develops SimSense, a new similarity dataset for German sense embeddings

Abstract

We introduce a neural network-based system of Word Sense Disambiguation (WSD) for German that is based on SenseFitting, a novel method for optimizing WSD. We outperform knowledge-based WSD methods by up to 25% F1-score and produce a new state-of-the-art on the German sense-annotated dataset WebCAGe. Our method uses three feature vectors consisting of a) sense, b) gloss, and c) relational vectors to represent target senses and to compare them with the vector centroids of sample contexts. Utilizing widely available word embeddings and lexical resources, we are able to compensate for the lower resource availability of German. SenseFitting builds upon the recently introduced semantic specialization procedure Attract-Repel, and leverages sense level semantic constraints from lexical-semantic networks (e.g. GermaNet) or online social dictionaries (e.g. Wiktionary) to produce high-quality sense embeddings from pre-trained word embeddings. We evaluate our sense embeddings with a new SimLex-999 based similarity dataset, called SimSense, that we developed for this work. We achieve results that outperform current lemma-based specialization methods for German, making them comparable to results achieved for English.