Beyond Empirical Risk Minimization: Local Structure Preserving Regularization for Improving Adversarial Robustness

TL;DR

This paper introduces a Local Structure Preserving regularization method that enhances adversarial robustness of neural networks by maintaining local data structure in the embedding space, improving defense against adversarial attacks.

Contribution

It proposes a novel regularization technique that leverages local data structure preservation to improve adversarial robustness, applicable with or without adversarial training.

Findings

Consistently improves adversarial robustness across multiple datasets

Enhances defense effectiveness without requiring extensive additional data

Outperforms several state-of-the-art methods in robustness metrics

Abstract

It is broadly known that deep neural networks are susceptible to being fooled by adversarial examples with perturbations imperceptible by humans. Various defenses have been proposed to improve adversarial robustness, among which adversarial training methods are most effective. However, most of these methods treat the training samples independently and demand a tremendous amount of samples to train a robust network, while ignoring the latent structural information among these samples. In this work, we propose a novel Local Structure Preserving (LSP) regularization, which aims to preserve the local structure of the input space in the learned embedding space. In this manner, the attacking effect of adversarial samples lying in the vicinity of clean samples can be alleviated. We show strong empirical evidence that with or without adversarial training, our method consistently improves the performance of adversarial robustness on several image classification datasets compared to the baselines and some state-of-the-art approaches, thus providing promising direction for future research.