AutoExtend: Extending Word Embeddings to Embeddings for Synsets and Lexemes

TL;DR

AutoExtend is a flexible system that extends existing word embeddings to include synsets and lexemes, maintaining the same vector space and achieving state-of-the-art results in word similarity and disambiguation tasks.

Contribution

It introduces a novel tensor-based method to derive embeddings for synsets and lexemes from any pre-existing word embeddings without additional training data.

Findings

Achieves state-of-the-art performance on word similarity tasks.

Effectively extends embeddings to synsets and lexemes.

Compatible with resources like WordNet and Freebase.

Abstract

We present \textit{AutoExtend}, a system to learn embeddings for synsets and lexemes. It is flexible in that it can take any word embeddings as input and does not need an additional training corpus. The synset/lexeme embeddings obtained live in the same vector space as the word embeddings. A sparse tensor formalization guarantees efficiency and parallelizability. We use WordNet as a lexical resource, but AutoExtend can be easily applied to other resources like Freebase. AutoExtend achieves state-of-the-art performance on word similarity and word sense disambiguation tasks.