Building on Huang et al. GlossBERT for Word Sense Disambiguation
Nikhil Patel, James Hale, Kanika Jindal, Apoorva Sharma, and Yichun Yu

TL;DR
This paper enhances the GlossBERT model for Word Sense Disambiguation by experimenting with dataset adjustments, ensemble techniques, and replacing BERT with BART and ALBERT, aiming to improve disambiguation accuracy.
Contribution
It introduces novel augmentations to the existing GlossBERT framework, including hyper-parameter tuning, ensemble methods, and model replacements, to advance WSD performance.
Findings
Improved disambiguation accuracy with dataset tweaking.
Enhanced results using ensemble methods.
Model replacements with BART and ALBERT show performance gains.
Abstract
We propose to take on the problem ofWord Sense Disambiguation (WSD). In language, words of the same form can take different meanings depending on context. While humans easily infer the meaning or gloss of such words by their context, machines stumble on this task.As such, we intend to replicated and expand upon the results of Huang et al.GlossBERT, a model which they design to disambiguate these words (Huang et al.,2019). Specifically, we propose the following augmentations: data-set tweaking(alpha hyper-parameter), ensemble methods, and replacement of BERT with BART andALBERT. The following GitHub repository contains all code used in this report, which extends on the code made available by Huang et al.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNatural Language Processing Techniques · Topic Modeling · Speech and dialogue systems
MethodsMulti-Head Attention · Attention Is All You Need · Linear Layer · Residual Connection · Layer Normalization · Attention Dropout · Linear Warmup With Linear Decay · Dense Connections · Refunds@Expedia|||How do I get a full refund from Expedia? · WordPiece
