Dynamically Sampled Nonlocal Gradients for Stronger Adversarial Attacks

TL;DR

This paper introduces DSNGD, a novel adversarial attack method that uses dynamically sampled nonlocal gradients to improve attack success rates and efficiency against deep neural networks.

Contribution

The paper proposes DSNGD, a new gradient sampling technique inspired by non-convex optimization, enhancing adversarial attack effectiveness and speed.

Findings

DSNGD achieves up to 27.1% higher success rates.

Attacks are on average 35% faster.

Provides more accurate global descent direction estimation.

Abstract

The vulnerability of deep neural networks to small and even imperceptible perturbations has become a central topic in deep learning research. Although several sophisticated defense mechanisms have been introduced, most were later shown to be ineffective. However, a reliable evaluation of model robustness is mandatory for deployment in safety-critical scenarios. To overcome this problem we propose a simple yet effective modification to the gradient calculation of state-of-the-art first-order adversarial attacks. Normally, the gradient update of an attack is directly calculated for the given data point. This approach is sensitive to noise and small local optima of the loss function. Inspired by gradient sampling techniques from non-convex optimization, we propose Dynamically Sampled Nonlocal Gradient Descent (DSNGD). DSNGD calculates the gradient direction of the adversarial attack as the weighted average over past gradients of the optimization history. Moreover, distribution hyperparameters that define the sampling operation are automatically learned during the optimization scheme. We empirically show that by incorporating this nonlocal gradient information, we are able to give a more accurate estimation of the global descent direction on noisy and non-convex loss surfaces. In addition, we show that DSNGD-based attacks are on average 35% faster while achieving 0.9% to 27.1% higher success rates compared to their gradient descent-based counterparts.