Query-Efficient Decision-based Black-Box Patch Attack

TL;DR

This paper introduces DevoPatch, a query-efficient decision-based patch attack method that models patches with key-points and uses evolutionary algorithms, significantly improving attack success rates and patch efficiency on various models.

Contribution

The paper proposes a novel decision-based patch attack method using paired key-points and evolutionary algorithms, advancing black-box attack efficiency and robustness evaluation.

Findings

DevoPatch outperforms state-of-the-art black-box patch attacks in success rate and patch size.

The method is effective on image classification and face verification tasks.

First evaluation of ViT and MLP models' vulnerability to decision-based patch attacks.

Abstract

Deep neural networks (DNNs) have been showed to be highly vulnerable to imperceptible adversarial perturbations. As a complementary type of adversary, patch attacks that introduce perceptible perturbations to the images have attracted the interest of researchers. Existing patch attacks rely on the architecture of the model or the probabilities of predictions and perform poorly in the decision-based setting, which can still construct a perturbation with the minimal information exposed -- the top-1 predicted label. In this work, we first explore the decision-based patch attack. To enhance the attack efficiency, we model the patches using paired key-points and use targeted images as the initialization of patches, and parameter optimizations are all performed on the integer domain. Then, we propose a differential evolutionary algorithm named DevoPatch for query-efficient decision-based patch attacks. Experiments demonstrate that DevoPatch outperforms the state-of-the-art black-box patch attacks in terms of patch area and attack success rate within a given query budget on image classification and face verification. Additionally, we conduct the vulnerability evaluation of ViT and MLP on image classification in the decision-based patch attack setting for the first time. Using DevoPatch, we can evaluate the robustness of models to black-box patch attacks. We believe this method could inspire the design and deployment of robust vision models based on various DNN architectures in the future.