Advancing Adversarial Robustness Through Adversarial Logit Update

TL;DR

This paper introduces Adversarial Logit Update (ALU), a novel principle for detecting and defending against adversarial attacks in neural networks by analyzing logit differences, improving robustness without additional training data.

Contribution

The paper proposes ALU, a new classification paradigm leveraging logit differences for adversarial robustness, applicable to pre-trained models without extra data or training.

Findings

Achieves superior robustness on CIFAR-10, CIFAR-100, and tiny-ImageNet datasets.

Effective against a wide range of adversarial attacks.

Does not require adversarial or additional training data.

Abstract

Deep Neural Networks are susceptible to adversarial perturbations. Adversarial training and adversarial purification are among the most widely recognized defense strategies. Although these methods have different underlying logic, both rely on absolute logit values to generate label predictions. In this study, we theoretically analyze the logit difference around successful adversarial attacks from a theoretical point of view and propose a new principle, namely Adversarial Logit Update (ALU), to infer adversarial sample's labels. Based on ALU, we introduce a new classification paradigm that utilizes pre- and post-purification logit differences for model's adversarial robustness boost. Without requiring adversarial or additional data for model training, our clean data synthesis model can be easily applied to various pre-trained models for both adversarial sample detection and ALU-based data classification. Extensive experiments on both CIFAR-10, CIFAR-100, and tiny-ImageNet datasets show that even with simple components, the proposed solution achieves superior robustness performance compared to state-of-the-art methods against a wide range of adversarial attacks. Our python implementation is submitted in our Supplementary document and will be published upon the paper's acceptance.