Stateful Detection of Black-Box Adversarial Attacks

TL;DR

This paper explores stateful defenses for black-box adversarial attacks, allowing detection of attack strategies by analyzing query histories, and introduces query blinding attacks to challenge such defenses.

Contribution

It introduces a novel stateful defense mechanism that detects adversarial query sequences and proposes query blinding attacks to circumvent these defenses.

Findings

Stateful defenses can identify adversarial query patterns.

Query blinding can bypass stateful detection methods.

Expanding to stateful systems offers new defense opportunities.

Abstract

The problem of adversarial examples, evasion attacks on machine learning classifiers, has proven extremely difficult to solve. This is true even when, as is the case in many practical settings, the classifier is hosted as a remote service and so the adversary does not have direct access to the model parameters. This paper argues that in such settings, defenders have a much larger space of actions than have been previously explored. Specifically, we deviate from the implicit assumption made by prior work that a defense must be a stateless function that operates on individual examples, and explore the possibility for stateful defenses. To begin, we develop a defense designed to detect the process of adversarial example generation. By keeping a history of the past queries, a defender can try to identify when a sequence of queries appears to be for the purpose of generating an adversarial example. We then introduce query blinding, a new class of attacks designed to bypass defenses that rely on such a defense approach. We believe that expanding the study of adversarial examples from stateless classifiers to stateful systems is not only more realistic for many black-box settings, but also gives the defender a much-needed advantage in responding to the adversary.