Transfer Learning and Augmentation for Word Sense Disambiguation

TL;DR

This paper presents a transfer learning and data augmentation pipeline that significantly improves Word Sense Disambiguation performance, achieving state-of-the-art results with a single model comparable to ensemble methods.

Contribution

It introduces a novel combination of transfer learning and data augmentation techniques specifically tailored for Word Sense Disambiguation.

Findings

Achieves state-of-the-art single model WSD performance

Matches top ensemble results in WSD

Demonstrates effectiveness of combined transfer learning and augmentation

Abstract

Many downstream NLP tasks have shown significant improvement through continual pre-training, transfer learning and multi-task learning. State-of-the-art approaches in Word Sense Disambiguation today benefit from some of these approaches in conjunction with information sources such as semantic relationships and gloss definitions contained within WordNet. Our work builds upon these systems and uses data augmentation along with extensive pre-training on various different NLP tasks and datasets. Our transfer learning and augmentation pipeline achieves state-of-the-art single model performance in WSD and is at par with the best ensemble results.