Definition Modeling: Learning to define word embeddings in natural language

TL;DR

This paper introduces definition modeling, a method to generate dictionary definitions from word embeddings using neural networks, providing a more direct evaluation of lexical semantics and revealing insights into embedding shortcomings.

Contribution

The paper proposes the task of definition modeling, develops neural network architectures for it, and demonstrates how this approach offers a transparent way to evaluate and analyze word embeddings.

Findings

Models controlling dependencies perform better

Character-level convolution enhances definitions

Errors reveal embedding limitations

Abstract

Distributed representations of words have been shown to capture lexical semantics, as demonstrated by their effectiveness in word similarity and analogical relation tasks. But, these tasks only evaluate lexical semantics indirectly. In this paper, we study whether it is possible to utilize distributed representations to generate dictionary definitions of words, as a more direct and transparent representation of the embeddings' semantics. We introduce definition modeling, the task of generating a definition for a given word and its embedding. We present several definition model architectures based on recurrent neural networks, and experiment with the models over multiple data sets. Our results show that a model that controls dependencies between the word being defined and the definition words performs significantly better, and that a character-level convolution layer designed to leverage morphology can complement word-level embeddings. Finally, an error analysis suggests that the errors made by a definition model may provide insight into the shortcomings of word embeddings.