Generating Transferable and Stealthy Adversarial Patch via Attention-guided Adversarial Inpainting

TL;DR

This paper introduces Adv-Inpainting, a two-stage method for creating stealthy, transferable adversarial patches that deceive face recognition models while maintaining natural appearance and camouflage.

Contribution

It proposes a novel two-stage attack combining style and identity feature extraction with attention-guided inpainting and refinement, improving stealthiness and transferability of adversarial patches.

Findings

Outperforms state-of-the-art adversarial patch attacks in transferability

Generates patches with higher visual quality and stealthiness

Achieves stronger deception of black-box face recognition models

Abstract

Adversarial patch attacks can fool the face recognition (FR) models via small patches. However, previous adversarial patch attacks often result in unnatural patterns that are easily noticeable. Generating transferable and stealthy adversarial patches that can efficiently deceive the black-box FR models while having good camouflage is challenging because of the huge stylistic difference between the source and target images. To generate transferable, natural-looking, and stealthy adversarial patches, we propose an innovative two-stage attack called Adv-Inpainting, which extracts style features and identity features from the attacker and target faces, respectively and then fills the patches with misleading and inconspicuous content guided by attention maps. In the first stage, we extract multi-scale style embeddings by a pyramid-like network and identity embeddings by a pretrained FR model and propose a novel Attention-guided Adaptive Instance Normalization layer (AAIN) to merge them via background-patch cross-attention maps. The proposed layer can adaptively fuse identity and style embeddings by fully exploiting priority contextual information. In the second stage, we design an Adversarial Patch Refinement Network (APR-Net) with a novel boundary variance loss, a spatial discounted reconstruction loss, and a perceptual loss to boost the stealthiness further. Experiments demonstrate that our attack can generate adversarial patches with improved visual quality, better stealthiness, and stronger transferability than state-of-the-art adversarial patch attacks and semantic attacks.