Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations

TL;DR

This paper introduces a novel data augmentation method called contextual augmentation, which replaces words with contextually predicted synonyms using a language model, improving text classification performance.

Contribution

The paper presents a new augmentation technique that leverages a label-conditional language model to replace words with contextually appropriate alternatives, enhancing classifier accuracy.

Findings

Improves performance of CNN and RNN classifiers

Effective across six different text classification tasks

Utilizes label-conditional language modeling for augmentation

Abstract

We propose a novel data augmentation for labeled sentences called contextual augmentation. We assume an invariance that sentences are natural even if the words in the sentences are replaced with other words with paradigmatic relations. We stochastically replace words with other words that are predicted by a bi-directional language model at the word positions. Words predicted according to a context are numerous but appropriate for the augmentation of the original words. Furthermore, we retrofit a language model with a label-conditional architecture, which allows the model to augment sentences without breaking the label-compatibility. Through the experiments for six various different text classification tasks, we demonstrate that the proposed method improves classifiers based on the convolutional or recurrent neural networks.