NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation

TL;DR

NL-Augmenter is a comprehensive, participatory framework for natural language data augmentation that supports diverse transformations and filters, aiding robustness evaluation and data diversity enhancement in NLP models.

Contribution

It introduces a new flexible framework with a large set of transformations and filters, enabling systematic data augmentation and robustness analysis in NLP.

Findings

Demonstrated improved robustness analysis of NLP models

Provided a publicly available repository of transformations and filters

Showcased the effectiveness of transformations in data augmentation

Abstract

Data augmentation is an important component in the robustness evaluation of models in natural language processing (NLP) and in enhancing the diversity of the data they are trained on. In this paper, we present NL-Augmenter, a new participatory Python-based natural language augmentation framework which supports the creation of both transformations (modifications to the data) and filters (data splits according to specific features). We describe the framework and an initial set of 117 transformations and 23 filters for a variety of natural language tasks. We demonstrate the efficacy of NL-Augmenter by using several of its transformations to analyze the robustness of popular natural language models. The infrastructure, datacards and robustness analysis results are available publicly on the NL-Augmenter repository (https://github.com/GEM-benchmark/NL-Augmenter).