Building on Huang et al. GlossBERT for Word Sense Disambiguation

TL;DR

This paper enhances the GlossBERT model for Word Sense Disambiguation by experimenting with dataset adjustments, ensemble techniques, and replacing BERT with BART and ALBERT, aiming to improve disambiguation accuracy.

Contribution

It introduces novel augmentations to the existing GlossBERT framework, including hyper-parameter tuning, ensemble methods, and model replacements, to advance WSD performance.

Findings

Improved disambiguation accuracy with dataset tweaking.

Enhanced results using ensemble methods.

Model replacements with BART and ALBERT show performance gains.

Abstract

We propose to take on the problem ofWord Sense Disambiguation (WSD). In language, words of the same form can take different meanings depending on context. While humans easily infer the meaning or gloss of such words by their context, machines stumble on this task.As such, we intend to replicated and expand upon the results of Huang et al.GlossBERT, a model which they design to disambiguate these words (Huang et al.,2019). Specifically, we propose the following augmentations: data-set tweaking(alpha hyper-parameter), ensemble methods, and replacement of BERT with BART andALBERT. The following GitHub repository contains all code used in this report, which extends on the code made available by Huang et al.