Black-box adversarial attacks using Evolution Strategies

TL;DR

This paper compares three evolution strategies for black-box adversarial attacks on neural networks, demonstrating their effectiveness and efficiency in fooling models without gradient information.

Contribution

It provides a comparative analysis of well-known evolution strategies for black-box adversarial attacks, highlighting their performance in different scenarios.

Findings

All algorithms can effectively fool neural networks

Some algorithms perform better in difficult attack scenarios

Black-box attacks require fewer queries with certain strategies

Abstract

In the last decade, deep neural networks have proven to be very powerful in computer vision tasks, starting a revolution in the computer vision and machine learning fields. However, deep neural networks, usually, are not robust to perturbations of the input data. In fact, several studies showed that slightly changing the content of the images can cause a dramatic decrease in the accuracy of the attacked neural network. Several methods able to generate adversarial samples make use of gradients, which usually are not available to an attacker in real-world scenarios. As opposed to this class of attacks, another class of adversarial attacks, called black-box adversarial attacks, emerged, which does not make use of information on the gradients, being more suitable for real-world attack scenarios. In this work, we compare three well-known evolution strategies on the generation of black-box adversarial attacks for image classification tasks. While our results show that the attacked neural networks can be, in most cases, easily fooled by all the algorithms under comparison, they also show that some black-box optimization algorithms may be better in "harder" setups, both in terms of attack success rate and efficiency (i.e., number of queries).