Overcoming Poor Word Embeddings with Word Definitions
Christopher Malon

TL;DR
This paper proposes a method for natural language understanding models to use natural language definitions to improve reasoning about rare or unseen words, bridging the gap caused by poor pretrained embeddings.
Contribution
It introduces a model that leverages natural language definitions to enhance understanding of rare words, addressing limitations of pretrained embeddings.
Findings
Model recovers most performance gap caused by untrained words
Using definitions improves reasoning about rare words
Definitions help compensate for poor embeddings
Abstract
Modern natural language understanding models depend on pretrained subword embeddings, but applications may need to reason about words that were never or rarely seen during pretraining. We show that examples that depend critically on a rarer word are more challenging for natural language inference models. Then we explore how a model could learn to use definitions, provided in natural text, to overcome this handicap. Our model's understanding of a definition is usually weaker than a well-modeled word embedding, but it recovers most of the performance gap from using a completely untrained word.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
