On Evaluation Protocols for Data Augmentation in a Limited Data Scenario

TL;DR

This paper critically examines data augmentation techniques in limited data scenarios, revealing that their effectiveness is mainly due to improved fine-tuning and that conversational agent-based augmentation outperforms classical methods.

Contribution

It demonstrates that classical data augmentation mainly enhances fine-tuning, and shows that zero- and few-shot augmentation via conversational agents can outperform traditional methods.

Findings

Classical data augmentation improves fine-tuning performance.

More fine-tuning before augmentation reduces its effectiveness.

Conversational agent-based augmentation yields better results.

Abstract

Textual data augmentation (DA) is a prolific field of study where novel techniques to create artificial data are regularly proposed, and that has demonstrated great efficiency on small data settings, at least for text classification tasks. In this paper, we challenge those results, showing that classical data augmentation (which modify sentences) is simply a way of performing better fine-tuning, and that spending more time doing so before applying data augmentation negates its effect. This is a significant contribution as it answers several questions that were left open in recent years, namely~: which DA technique performs best (all of them as long as they generate data close enough to the training set, as to not impair training) and why did DA show positive results (facilitates training of network). We further show that zero- and few-shot DA via conversational agents such as ChatGPT or LLama2 can increase performances, confirming that this form of data augmentation is preferable to classical methods.