Adaptive Clustering of Robust Semantic Representations for Adversarial Image Purification

TL;DR

This paper introduces a model-agnostic, semantic clustering-based method for adversarial image purification that improves robustness and accuracy against unseen attacks on datasets like CIFAR-10 and ImageNet-10.

Contribution

It proposes a novel semantic clustering and dictionary learning approach for adversarial defense that enhances generalization and transferability across models.

Findings

Improved accuracy by over 10% on CIFAR-10 and ImageNet-10.

Effective purification of adversarial images by reconstructing high-frequency components.

Model-agnostic method that generalizes to unseen adversarial attacks.

Abstract

Deep Learning models are highly susceptible to adversarial manipulations that can lead to catastrophic consequences. One of the most effective methods to defend against such disturbances is adversarial training but at the cost of generalization of unseen attacks and transferability across models. In this paper, we propose a robust defense against adversarial attacks, which is model agnostic and generalizable to unseen adversaries. Initially, with a baseline model, we extract the latent representations for each class and adaptively cluster the latent representations that share a semantic similarity. We obtain the distributions for the clustered latent representations and from their originating images, we learn semantic reconstruction dictionaries (SRD). We adversarially train a new model constraining the latent space representation to minimize the distance between the adversarial latent representation and the true cluster distribution. To purify the image, we decompose the input into low and high-frequency components. The high-frequency component is reconstructed based on the most adequate SRD from the clean dataset. In order to evaluate the most adequate SRD, we rely on the distance between robust latent representations and semantic cluster distributions. The output is a purified image with no perturbation. Image purification on CIFAR-10 and ImageNet-10 using our proposed method improved the accuracy by more than 10% compared to state-of-the-art results.