Understanding the Robustness of Randomized Feature Defense Against Query-Based Adversarial Attacks

TL;DR

This paper introduces a simple, model-agnostic defense mechanism that adds random noise to hidden features during inference to improve robustness against black-box adversarial attacks without retraining.

Contribution

It proposes a lightweight, noise-based defense method that enhances black-box attack resilience without impacting model accuracy or requiring adversarial training.

Findings

Effective against multiple black-box attack types

Minimal impact on model accuracy

Applicable to pre-trained models without retraining

Abstract

Recent works have shown that deep neural networks are vulnerable to adversarial examples that find samples close to the original image but can make the model misclassify. Even with access only to the model's output, an attacker can employ black-box attacks to generate such adversarial examples. In this work, we propose a simple and lightweight defense against black-box attacks by adding random noise to hidden features at intermediate layers of the model at inference time. Our theoretical analysis confirms that this method effectively enhances the model's resilience against both score-based and decision-based black-box attacks. Importantly, our defense does not necessitate adversarial training and has minimal impact on accuracy, rendering it applicable to any pre-trained model. Our analysis also reveals the significance of selectively adding noise to different parts of the model based on the gradient of the adversarial objective function, which can be varied during the attack. We demonstrate the robustness of our defense against multiple black-box attacks through extensive empirical experiments involving diverse models with various architectures.