Distributional Modeling for Location-Aware Adversarial Patches

TL;DR

This paper introduces DOPatch, a novel distributional approach for optimizing location-aware adversarial patches, enhancing attack efficiency, diversity, and robustness in physical-world adversarial scenarios.

Contribution

The paper proposes a distribution-optimized method for adversarial patch placement, enabling efficient black-box attacks and improved robustness through distributional modeling and training.

Findings

DOPatch outperforms existing methods in attack success and efficiency.

Distributional prior enables effective black-box attacks on unseen models.

DOP-DMAT improves model robustness against location-aware adversarial patches.

Abstract

Adversarial patch is one of the important forms of performing adversarial attacks in the physical world. To improve the naturalness and aggressiveness of existing adversarial patches, location-aware patches are proposed, where the patch's location on the target object is integrated into the optimization process to perform attacks. Although it is effective, efficiently finding the optimal location for placing the patches is challenging, especially under the black-box attack settings. In this paper, we propose the Distribution-Optimized Adversarial Patch (DOPatch), a novel method that optimizes a multimodal distribution of adversarial locations instead of individual ones. DOPatch has several benefits: Firstly, we find that the locations' distributions across different models are pretty similar, and thus we can achieve efficient query-based attacks to unseen models using a distributional prior optimized on a surrogate model. Secondly, DOPatch can generate diverse adversarial samples by characterizing the distribution of adversarial locations. Thus we can improve the model's robustness to location-aware patches via carefully designed Distributional-Modeling Adversarial Training (DOP-DMAT). We evaluate DOPatch on various face recognition and image recognition tasks and demonstrate its superiority and efficiency over existing methods. We also conduct extensive ablation studies and analyses to validate the effectiveness of our method and provide insights into the distribution of adversarial locations.