Self-Supervised Representation Learning for Adversarial Attack Detection

TL;DR

This paper introduces a self-supervised learning framework for detecting adversarial attacks that does not require extensive labeled data and achieves state-of-the-art results across various image datasets.

Contribution

The paper presents a novel self-supervised representation learning approach with prototype clustering and a parallel axial-attention encoder for improved adversarial attack detection.

Findings

Outperforms benchmark self-supervised vision models.

Achieves state-of-the-art detection accuracy.

Effective across diverse image datasets.

Abstract

Supervised learning-based adversarial attack detection methods rely on a large number of labeled data and suffer significant performance degradation when applying the trained model to new domains. In this paper, we propose a self-supervised representation learning framework for the adversarial attack detection task to address this drawback. Firstly, we map the pixels of augmented input images into an embedding space. Then, we employ the prototype-wise contrastive estimation loss to cluster prototypes as latent variables. Additionally, drawing inspiration from the concept of memory banks, we introduce a discrimination bank to distinguish and learn representations for each individual instance that shares the same or a similar prototype, establishing a connection between instances and their associated prototypes. We propose a parallel axial-attention (PAA)-based encoder to facilitate the training process by parallel training over height- and width-axis of attention maps. Experimental results show that, compared to various benchmark self-supervised vision learning models and supervised adversarial attack detection methods, the proposed model achieves state-of-the-art performance on the adversarial attack detection task across a wide range of images.