Reducing and Exploiting Data Augmentation Noise through Meta Reweighting Contrastive Learning for Text Classification

TL;DR

This paper introduces a novel meta reweighting framework that leverages contrastive learning to improve text classification by effectively utilizing and refining augmented data based on their quality.

Contribution

The paper proposes a new framework combining meta learning and contrastive learning to reweight and refine augmented samples, enhancing model performance on text classification tasks.

Findings

Achieves up to 4.4% absolute improvement on GLUE datasets.

Effectively cooperates with existing models like RoBERTa and Text-CNN.

Provides in-depth analysis of network components' contributions.

Abstract

Data augmentation has shown its effectiveness in resolving the data-hungry problem and improving model's generalization ability. However, the quality of augmented data can be varied, especially compared with the raw/original data. To boost deep learning models' performance given augmented data/samples in text classification tasks, we propose a novel framework, which leverages both meta learning and contrastive learning techniques as parts of our design for reweighting the augmented samples and refining their feature representations based on their quality. As part of the framework, we propose novel weight-dependent enqueue and dequeue algorithms to utilize augmented samples' weight/quality information effectively. Through experiments, we show that our framework can reasonably cooperate with existing deep learning models (e.g., RoBERTa-base and Text-CNN) and augmentation techniques (e.g., Wordnet and Easydata) for specific supervised learning tasks. Experiment results show that our framework achieves an average of 1.6%, up to 4.3% absolute improvement on Text-CNN encoders and an average of 1.4%, up to 4.4% absolute improvement on RoBERTa-base encoders on seven GLUE benchmark datasets compared with the best baseline. We present an indepth analysis of our framework design, revealing the non-trivial contributions of our network components. Our code is publicly available for better reproducibility.