MAT: Mixed-Strategy Game of Adversarial Training in Fine-tuning

TL;DR

This paper introduces MAT, a novel mixed-strategy adversarial training method for fine-tuning large language models, which improves robustness and generalization by leveraging game theory and Nash equilibrium concepts.

Contribution

The paper proposes a new adversarial training algorithm based on mixed strategies and Nash equilibrium, enhancing model performance beyond pure-strategy methods.

Findings

MAT outperforms state-of-the-art methods on GLUE and ANLI benchmarks.

MAT improves model robustness and generalization.

Extensive experiments validate the effectiveness of the proposed approach.

Abstract

Fine-tuning large-scale pre-trained language models has been demonstrated effective for various natural language processing (NLP) tasks. Previous studies have established that incorporating adversarial training during the fine-tuning stage can significantly enhance model generalization and robustness. However, from the perspective of game theory, such utilizations of adversarial training correspond to pure-strategy games, which are inherently limited in terms of the scope of their strategies, thereby still having room for improvement. In order to push the performance boundaries, we propose a novel Mixed-strategy Adversarial Training algorithm (MAT). Methodologically, we derive the Nash equilibrium of a mixed-strategy game for adversarial training using Entropy Mirror Descent to establish MAT by sampling method. To verify the effectiveness of MAT, we conducted extensive benchmark experiments on large-scale pre-trained models, such as BERT and RoBERTa. MAT significantly outperforms the state-of-the-art methods on both the GLUE and ANLI benchmarks in terms of generalization and robustness.