Certified Robustness to Programmable Transformations in LSTMs

TL;DR

This paper introduces a method to certify and train LSTM models that are robust against complex, programmatically defined string transformations, improving adversarial robustness in NLP tasks.

Contribution

It presents a novel certification approach for LSTMs against large, programmatic string perturbations and demonstrates its effectiveness in training more robust models.

Findings

Models trained with our approach are more robust to string transformations.

Our certification method achieves high accuracy in verifying robustness.

The approach handles intractably large perturbation spaces.

Abstract

Deep neural networks for natural language processing are fragile in the face of adversarial examples -- small input perturbations, like synonym substitution or word duplication, which cause a neural network to change its prediction. We present an approach to certifying the robustness of LSTMs (and extensions of LSTMs) and training models that can be efficiently certified. Our approach can certify robustness to intractably large perturbation spaces defined programmatically in a language of string transformations. Our evaluation shows that (1) our approach can train models that are more robust to combinations of string transformations than those produced using existing techniques; (2) our approach can show high certification accuracy of the resulting models.