A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologies

TL;DR

This survey comprehensively reviews robust adversarial training in pattern recognition, covering fundamentals, theories, methodologies, and connections to traditional learning, aiming to deepen understanding and guide future research in defending neural networks against adversarial attacks.

Contribution

It provides a systematic, structured overview of adversarial training, including a unified theoretical framework, visualizations, and connections to classical learning theories, which were previously lacking.

Findings

Unified theoretical framework for adversarial training

Visualizations explaining robustness mechanisms

Connections between adversarial training and traditional learning theories

Abstract

In the last a few decades, deep neural networks have achieved remarkable success in machine learning, computer vision, and pattern recognition. Recent studies however show that neural networks (both shallow and deep) may be easily fooled by certain imperceptibly perturbed input samples called adversarial examples. Such security vulnerability has resulted in a large body of research in recent years because real-world threats could be introduced due to vast applications of neural networks. To address the robustness issue to adversarial examples particularly in pattern recognition, robust adversarial training has become one mainstream. Various ideas, methods, and applications have boomed in the field. Yet, a deep understanding of adversarial training including characteristics, interpretations, theories, and connections among different models has still remained elusive. In this paper, we present a comprehensive survey trying to offer a systematic and structured investigation on robust adversarial training in pattern recognition. We start with fundamentals including definition, notations, and properties of adversarial examples. We then introduce a unified theoretical framework for defending against adversarial samples - robust adversarial training with visualizations and interpretations on why adversarial training can lead to model robustness. Connections will be also established between adversarial training and other traditional learning theories. After that, we summarize, review, and discuss various methodologies with adversarial attack and defense/training algorithms in a structured way. Finally, we present analysis, outlook, and remarks of adversarial training.