IAE: Irony-based Adversarial Examples for Sentiment Analysis Systems

TL;DR

This paper introduces Irony-based Adversarial Examples (IAE) that transform straightforward sentences into ironic ones to attack sentiment analysis systems, revealing their vulnerability to irony-based adversarial manipulation.

Contribution

We propose a novel irony-based method for generating adversarial text without relying on existing irony corpora, highlighting system vulnerabilities.

Findings

State-of-the-art models' performance drops significantly under IAE attacks.

Humans are less affected by irony in text compared to NLP systems.

IAE effectively exploits irony to create challenging adversarial examples.

Abstract

Adversarial examples, which are inputs deliberately perturbed with imperceptible changes to induce model errors, have raised serious concerns for the reliability and security of deep neural networks (DNNs). While adversarial attacks have been extensively studied in continuous data domains such as images, the discrete nature of text presents unique challenges. In this paper, we propose Irony-based Adversarial Examples (IAE), a method that transforms straightforward sentences into ironic ones to create adversarial text. This approach exploits the rhetorical device of irony, where the intended meaning is opposite to the literal interpretation, requiring a deeper understanding of context to detect. The IAE method is particularly challenging due to the need to accurately locate evaluation words, substitute them with appropriate collocations, and expand the text with suitable ironic elements while maintaining semantic coherence. Our research makes the following key contributions: (1) We introduce IAE, a strategy for generating textual adversarial examples using irony. This method does not rely on pre-existing irony corpora, making it a versatile tool for creating adversarial text in various NLP tasks. (2) We demonstrate that the performance of several state-of-the-art deep learning models on sentiment analysis tasks significantly deteriorates when subjected to IAE attacks. This finding underscores the susceptibility of current NLP systems to adversarial manipulation through irony. (3) We compare the impact of IAE on human judgment versus NLP systems, revealing that humans are less susceptible to the effects of irony in text.