MUSE: Modularizing Unsupervised Sense Embeddings

TL;DR

MUSE introduces a modular, unsupervised system for sense embeddings that separates sense representation learning from sense selection, achieving state-of-the-art results efficiently.

Contribution

It presents a novel modular framework that allows independent optimization of sense representation and selection, enabling linear-time sense selection and improved robustness.

Findings

Achieves state-of-the-art on synonym selection

Outperforms previous models on contextual word similarity

Uses reinforcement learning for joint module training

Abstract

This paper proposes to address the word sense ambiguity issue in an unsupervised manner, where word sense representations are learned along a word sense selection mechanism given contexts. Prior work focused on designing a single model to deliver both mechanisms, and thus suffered from either coarse-grained representation learning or inefficient sense selection. The proposed modular approach, MUSE, implements flexible modules to optimize distinct mechanisms, achieving the first purely sense-level representation learning system with linear-time sense selection. We leverage reinforcement learning to enable joint training on the proposed modules, and introduce various exploration techniques on sense selection for better robustness. The experiments on benchmark data show that the proposed approach achieves the state-of-the-art performance on synonym selection as well as on contextual word similarities in terms of MaxSimC.