Improving Adversarial Robustness via Phase and Amplitude-aware Prompting

TL;DR

This paper introduces a novel defense method called Phase and Amplitude-aware Prompting (PAP) that enhances adversarial robustness of neural networks by leveraging phase and amplitude spectra in prompts, leading to improved performance.

Contribution

The paper proposes a new prompt-based defense utilizing phase and amplitude spectra, which better focus on semantic patterns and improve adversarial robustness.

Findings

PAP significantly improves robustness against adversarial attacks.

The method outperforms existing prompt-based defenses.

Experimental results validate the effectiveness of phase and amplitude prompts.

Abstract

Deep neural networks are found to be vulnerable to adversarial perturbations. The prompt-based defense has been increasingly studied due to its high efficiency. However, existing prompt-based defenses mainly exploited mixed prompt patterns, where critical patterns closely related to object semantics lack sufficient focus. The phase and amplitude spectra have been proven to be highly related to specific semantic patterns and crucial for robustness. To this end, in this paper, we propose a Phase and Amplitude-aware Prompting (PAP) defense. Specifically, we construct phase-level and amplitude-level prompts for each class, and adjust weights for prompting according to the model's robust performance under these prompts during training. During testing, we select prompts for each image using its predicted label to obtain the prompted image, which is inputted to the model to get the final prediction. Experimental results demonstrate the effectiveness of our method.