SMSMix: Sense-Maintained Sentence Mixup for Word Sense Disambiguation

TL;DR

This paper introduces SMSMix, a novel data augmentation technique for Word Sense Disambiguation that preserves the sense of target words during sentence mixing, improving recognition of rare senses.

Contribution

It proposes the first sense-preserving mixup method for NLP, enhancing training data for better rare sense recognition in WSD tasks.

Findings

Improves recognition of rare word senses in WSD

Effectively maintains target word sense during augmentation

Enhances training data diversity for better model performance

Abstract

Word Sense Disambiguation (WSD) is an NLP task aimed at determining the correct sense of a word in a sentence from discrete sense choices. Although current systems have attained unprecedented performances for such tasks, the nonuniform distribution of word senses during training generally results in systems performing poorly on rare senses. To this end, we consider data augmentation to increase the frequency of these least frequent senses (LFS) to reduce the distributional bias of senses during training. We propose Sense-Maintained Sentence Mixup (SMSMix), a novel word-level mixup method that maintains the sense of a target word. SMSMix smoothly blends two sentences using mask prediction while preserving the relevant span determined by saliency scores to maintain a specific word's sense. To the best of our knowledge, this is the first attempt to apply mixup in NLP while preserving the meaning of a specific word. With extensive experiments, we validate that our augmentation method can effectively give more information about rare senses during training with maintained target sense label.