Learning with Multiplicative Perturbations

TL;DR

This paper introduces xAT and xVAT, novel adversarial training methods using multiplicative perturbations for more interpretable and efficient robust DNN training, outperforming traditional additive approaches.

Contribution

The paper proposes multiplicative perturbations for adversarial training, enabling more perceptible, interpretable, and faster robust DNN training compared to additive methods.

Findings

xAT and xVAT match or outperform state-of-the-art accuracy.

They are approximately 30% faster than additive methods.

Resulting DNNs show distinct weight distributions.

Abstract

Adversarial Training (AT) and Virtual Adversarial Training (VAT) are the regularization techniques that train Deep Neural Networks (DNNs) with adversarial examples generated by adding small but worst-case perturbations to input examples. In this paper, we propose xAT and xVAT, new adversarial training algorithms, that generate \textbf{multiplicative} perturbations to input examples for robust training of DNNs. Such perturbations are much more perceptible and interpretable than their \textbf{additive} counterparts exploited by AT and VAT. Furthermore, the multiplicative perturbations can be generated transductively or inductively while the standard AT and VAT only support a transductive implementation. We conduct a series of experiments that analyze the behavior of the multiplicative perturbations and demonstrate that xAT and xVAT match or outperform state-of-the-art classification accuracies across multiple established benchmarks while being about 30\% faster than their additive counterparts. Furthermore, the resulting DNNs also demonstrate distinct weight distributions.