Joint Word Representation Learning using a Corpus and a Semantic Lexicon

TL;DR

This paper introduces a joint learning approach that combines corpus-based co-occurrence data with semantic lexicon constraints to produce improved word representations for NLP tasks.

Contribution

It proposes a novel method that integrates semantic lexicon information into word embedding learning, enhancing the quality of representations over previous approaches.

Findings

Significantly outperforms previous methods on semantic similarity tasks.

Achieves better results on word analogy benchmarks.

Effectively incorporates semantic relations into embeddings.

Abstract

Methods for learning word representations using large text corpora have received much attention lately due to their impressive performance in numerous natural language processing (NLP) tasks such as, semantic similarity measurement, and word analogy detection. Despite their success, these data-driven word representation learning methods do not consider the rich semantic relational structure between words in a co-occurring context. On the other hand, already much manual effort has gone into the construction of semantic lexicons such as the WordNet that represent the meanings of words by defining the various relationships that exist among the words in a language. We consider the question, can we improve the word representations learnt using a corpora by integrating the knowledge from semantic lexicons?. For this purpose, we propose a joint word representation learning method that simultaneously predicts the co-occurrences of two words in a sentence subject to the relational constrains given by the semantic lexicon. We use relations that exist between words in the lexicon to regularize the word representations learnt from the corpus. Our proposed method statistically significantly outperforms previously proposed methods for incorporating semantic lexicons into word representations on several benchmark datasets for semantic similarity and word analogy.