Word Sense Induction with Neural biLM and Symmetric Patterns

TL;DR

This paper introduces a novel approach for Word Sense Induction using a recurrent language model combined with dynamic symmetric patterns, significantly improving state-of-the-art results on the SemEval 2013 WSI task.

Contribution

It replaces ngram-based models with a recurrent language model and introduces dynamic symmetric patterns for better sense induction.

Findings

Surpasses previous state-of-the-art on SemEval 2013 WSI

Recurrent LM provides more accurate substitute vectors

Dynamic symmetric patterns enhance query effectiveness

Abstract

An established method for Word Sense Induction (WSI) uses a language model to predict probable substitutes for target words, and induces senses by clustering these resulting substitute vectors. We replace the ngram-based language model (LM) with a recurrent one. Beyond being more accurate, the use of the recurrent LM allows us to effectively query it in a creative way, using what we call dynamic symmetric patterns. The combination of the RNN-LM and the dynamic symmetric patterns results in strong substitute vectors for WSI, allowing to surpass the current state-of-the-art on the SemEval 2013 WSI shared task by a large margin.