IAP: Invisible Adversarial Patch Attack through Perceptibility-Aware Localization and Perturbation Optimization

TL;DR

IAP introduces a novel method for generating highly invisible adversarial patches that are effective against models and defenses, leveraging perceptibility-aware localization and perturbation optimization to enhance stealthiness.

Contribution

The paper proposes a new attack framework that produces more imperceptible adversarial patches by combining perceptibility-aware localization with optimized perturbation strategies.

Findings

Achieves high attack success rates in targeted scenarios.

Produces patches that are significantly more invisible to humans.

Effectively bypasses several state-of-the-art patch defenses.

Abstract

Despite modifying only a small localized input region, adversarial patches can drastically change the prediction of computer vision models. However, prior methods either cannot perform satisfactorily under targeted attack scenarios or fail to produce contextually coherent adversarial patches, causing them to be easily noticeable by human examiners and insufficiently stealthy against automatic patch defenses. In this paper, we introduce IAP, a novel attack framework that generates highly invisible adversarial patches based on perceptibility-aware localization and perturbation optimization schemes. Specifically, IAP first searches for a proper location to place the patch by leveraging classwise localization and sensitivity maps, balancing the susceptibility of patch location to both victim model prediction and human visual system, then employs a perceptibility-regularized adversarial loss and a gradient update rule that prioritizes color constancy for optimizing invisible perturbations. Comprehensive experiments across various image benchmarks and model architectures demonstrate that IAP consistently achieves competitive attack success rates in targeted settings with significantly improved patch invisibility compared to existing baselines. In addition to being highly imperceptible to humans, IAP is shown to be stealthy enough to render several state-of-the-art patch defenses ineffective.