Defensive Dual Masking for Robust Adversarial Defense

TL;DR

The paper presents Defensive Dual Masking (DDM), a novel adversarial defense method for NLP models that uses strategic masking during training and inference to improve robustness against adversarial attacks.

Contribution

Introduces DDM, a new adversarial defense technique employing strategic masking during training and inference to enhance NLP model robustness.

Findings

DDM outperforms existing defenses on multiple benchmarks.

DDM improves robustness of Large Language Models against adversarial attacks.

Empirical results show increased accuracy and resilience.

Abstract

The field of textual adversarial defenses has gained considerable attention in recent years due to the increasing vulnerability of natural language processing (NLP) models to adversarial attacks, which exploit subtle perturbations in input text to deceive models. This paper introduces the Defensive Dual Masking (DDM) algorithm, a novel approach designed to enhance model robustness against such attacks. DDM utilizes a unique adversarial training strategy where [MASK] tokens are strategically inserted into training samples to prepare the model to handle adversarial perturbations more effectively. During inference, potentially adversarial tokens are dynamically replaced with [MASK] tokens to neutralize potential threats while preserving the core semantics of the input. The theoretical foundation of our approach is explored, demonstrating how the selective masking mechanism strengthens the model's ability to identify and mitigate adversarial manipulations. Our empirical evaluation across a diverse set of benchmark datasets and attack mechanisms consistently shows that DDM outperforms state-of-the-art defense techniques, improving model accuracy and robustness. Moreover, when applied to Large Language Models (LLMs), DDM also enhances their resilience to adversarial attacks, providing a scalable defense mechanism for large-scale NLP applications.