Simultaneously Optimizing Perturbations and Positions for Black-box Adversarial Patch Attacks

TL;DR

This paper introduces a reinforcement learning-based method to simultaneously optimize the position and perturbations of adversarial patches, significantly improving attack success rates and query efficiency in black-box settings across face recognition and traffic sign tasks.

Contribution

It proposes a novel approach that jointly optimizes patch position and perturbations using reinforcement learning, enhancing black-box adversarial attack effectiveness.

Findings

Improved attack success rate on face recognition models.

Enhanced query efficiency in black-box attacks.

Validated effectiveness in physical environments and traffic sign recognition.

Abstract

Adversarial patch is an important form of real-world adversarial attack that brings serious risks to the robustness of deep neural networks. Previous methods generate adversarial patches by either optimizing their perturbation values while fixing the pasting position or manipulating the position while fixing the patch's content. This reveals that the positions and perturbations are both important to the adversarial attack. For that, in this paper, we propose a novel method to simultaneously optimize the position and perturbation for an adversarial patch, and thus obtain a high attack success rate in the black-box setting. Technically, we regard the patch's position, the pre-designed hyper-parameters to determine the patch's perturbations as the variables, and utilize the reinforcement learning framework to simultaneously solve for the optimal solution based on the rewards obtained from the target model with a small number of queries. Extensive experiments are conducted on the Face Recognition (FR) task, and results on four representative FR models show that our method can significantly improve the attack success rate and query efficiency. Besides, experiments on the commercial FR service and physical environments confirm its practical application value. We also extend our method to the traffic sign recognition task to verify its generalization ability.