Adversarial Robustness through Dynamic Ensemble Learning

TL;DR

This paper introduces ARDEL, a dynamic ensemble learning approach that significantly improves the adversarial robustness of pre-trained language models by adaptively reconfiguring ensembles based on input and attack patterns.

Contribution

ARDEL is a novel ensemble scheme that dynamically adjusts model configurations using a meta-model and adversarial detection, enhancing robustness against attacks.

Findings

ARDEL outperforms existing defenses in robustness tests.

Dynamic reconfiguration reduces attack success rates.

Maintains higher accuracy under adversarial conditions.

Abstract

Adversarial attacks pose a significant threat to the reliability of pre-trained language models (PLMs) such as GPT, BERT, RoBERTa, and T5. This paper presents Adversarial Robustness through Dynamic Ensemble Learning (ARDEL), a novel scheme designed to enhance the robustness of PLMs against such attacks. ARDEL leverages the diversity of multiple PLMs and dynamically adjusts the ensemble configuration based on input characteristics and detected adversarial patterns. Key components of ARDEL include a meta-model for dynamic weighting, an adversarial pattern detection module, and adversarial training with regularization techniques. Comprehensive evaluations using standardized datasets and various adversarial attack scenarios demonstrate that ARDEL significantly improves robustness compared to existing methods. By dynamically reconfiguring the ensemble to prioritize the most robust models for each input, ARDEL effectively reduces attack success rates and maintains higher accuracy under adversarial conditions. This work contributes to the broader goal of developing more secure and trustworthy AI systems for real-world NLP applications, offering a practical and scalable solution to enhance adversarial resilience in PLMs.