Advocating for Multiple Defense Strategies against Adversarial Examples

TL;DR

This paper analyzes the limitations of current defense strategies against different types of adversarial examples in neural networks, highlighting the need for multiple, combined defenses.

Contribution

It provides a geometric validation of the ineffectiveness of single-strategy defenses and reviews existing multi-strategy defense approaches with empirical insights.

Findings

Single-strategy defenses perform poorly across different attack norms.

Empirical evidence shows the benefit of combining multiple defense strategies.

Open questions remain on optimal multi-strategy defense design.

Abstract

It has been empirically observed that defense mechanisms designed to protect neural networks against $\ell_\infty$ adversarial examples offer poor performance against $\ell_2$ adversarial examples and vice versa. In this paper we conduct a geometrical analysis that validates this observation. Then, we provide a number of empirical insights to illustrate the effect of this phenomenon in practice. Then, we review some of the existing defense mechanism that attempts to defend against multiple attacks by mixing defense strategies. Thanks to our numerical experiments, we discuss the relevance of this method and state open questions for the adversarial examples community.