Meta-Learning with Variational Semantic Memory for Word Sense Disambiguation

TL;DR

This paper introduces a variational semantic memory model for word sense disambiguation that enhances few-shot learning by leveraging prior experiences, hierarchical inference, and adaptive memory updates, significantly improving performance in data-scarce scenarios.

Contribution

It proposes a novel semantic memory model with hierarchical variational inference and hypernetwork-based memory updates for improved few-shot WSD.

Findings

Achieves state-of-the-art results in few-shot WSD

Effective in one-shot learning scenarios

Produces meaningful sense prototypes

Abstract

A critical challenge faced by supervised word sense disambiguation (WSD) is the lack of large annotated datasets with sufficient coverage of words in their diversity of senses. This inspired recent research on few-shot WSD using meta-learning. While such work has successfully applied meta-learning to learn new word senses from very few examples, its performance still lags behind its fully supervised counterpart. Aiming to further close this gap, we propose a model of semantic memory for WSD in a meta-learning setting. Semantic memory encapsulates prior experiences seen throughout the lifetime of the model, which aids better generalization in limited data settings. Our model is based on hierarchical variational inference and incorporates an adaptive memory update rule via a hypernetwork. We show our model advances the state of the art in few-shot WSD, supports effective learning in extremely data scarce (e.g. one-shot) scenarios and produces meaning prototypes that capture similar senses of distinct words.