Towards better substitution-based word sense induction

TL;DR

This paper improves word sense induction by extending substitute-based clustering methods to support dynamic clusters, integrating BERT, and providing interpretability and error analysis for better unsupervised sense clustering.

Contribution

It introduces a dynamic clustering extension for substitute-based WSI, adapts BERT for improved performance, and offers interpretability and detailed error analysis.

Findings

Extends substitute-based WSI to support dynamic clusters.

Adapts BERT to enhance WSI performance.

Provides interpretability and error analysis for WSI.

Abstract

Word sense induction (WSI) is the task of unsupervised clustering of word usages within a sentence to distinguish senses. Recent work obtain strong results by clustering lexical substitutes derived from pre-trained RNN language models (ELMo). Adapting the method to BERT improves the scores even further. We extend the previous method to support a dynamic rather than a fixed number of clusters as supported by other prominent methods, and propose a method for interpreting the resulting clusters by associating them with their most informative substitutes. We then perform extensive error analysis revealing the remaining sources of errors in the WSI task. Our code is available at https://github.com/asafamr/bertwsi.