Learning Robust Representations of Text

TL;DR

This paper introduces a regularization technique inspired by computer vision to enhance the robustness of neural network models against noise and adversarial attacks in text processing tasks.

Contribution

It proposes a novel regularization method that reduces input sensitivity, improving robustness of text models beyond traditional dropout techniques.

Findings

Outperforms baseline and dropout in noisy input scenarios

Achieves higher accuracy on out-of-domain data

Demonstrates robustness across multiple sentiment datasets

Abstract

Deep neural networks have achieved remarkable results across many language processing tasks, however these methods are highly sensitive to noise and adversarial attacks. We present a regularization based method for limiting network sensitivity to its inputs, inspired by ideas from computer vision, thus learning models that are more robust. Empirical evaluation over a range of sentiment datasets with a convolutional neural network shows that, compared to a baseline model and the dropout method, our method achieves superior performance over noisy inputs and out-of-domain data.