Attacking interpretable NLP systems

TL;DR

This paper presents AdvChar, a black-box attack method that subtly modifies text inputs to deceive interpretable NLP systems while maintaining semantic similarity and interpretation, exposing vulnerabilities in model trust.

Contribution

Introduces AdvChar, a novel character-level attack that effectively misleads interpretable NLP models with minimal text modifications, highlighting security concerns.

Findings

AdvChar reduces model accuracy significantly with minimal text changes.

The attack maintains high similarity between original and adversarial inputs.

It successfully fools multiple NLP and interpretation models.

Abstract

Studies have shown that machine learning systems are vulnerable to adversarial examples in theory and practice. Where previous attacks have focused mainly on visual models that exploit the difference between human and machine perception, text-based models have also fallen victim to these attacks. However, these attacks often fail to maintain the semantic meaning of the text and similarity. This paper introduces AdvChar, a black-box attack on Interpretable Natural Language Processing Systems, designed to mislead the classifier while keeping the interpretation similar to benign inputs, thus exploiting trust in system transparency. AdvChar achieves this by making less noticeable modifications to text input, forcing the deep learning classifier to make incorrect predictions and preserve the original interpretation. We use an interpretation-focused scoring approach to determine the most critical tokens that, when changed, can cause the classifier to misclassify the input. We apply simple character-level modifications to measure the importance of tokens, minimizing the difference between the original and new text while generating adversarial interpretations similar to benign ones. We thoroughly evaluated AdvChar by testing it against seven NLP models and three interpretation models using benchmark datasets for the classification task. Our experiments show that AdvChar can significantly reduce the prediction accuracy of current deep learning models by altering just two characters on average in input samples.