Effects of Loss Functions And Target Representations on Adversarial Robustness

TL;DR

This paper investigates how different loss functions and target representations, such as mean-squared error and codewords, can significantly enhance neural network robustness against adversarial attacks, outperforming traditional models.

Contribution

The study introduces novel training strategies using alternative loss functions and target representations, demonstrating substantial improvements in adversarial robustness.

Findings

Increased accuracy against untargeted attacks up to 98.7%

Reduced targeted attack success rates up to 99.8%

Smaller Lipschitz bounds indicating lower sensitivity to perturbations

Abstract

Understanding and evaluating the robustness of neural networks under adversarial settings is a subject of growing interest. Attacks proposed in the literature usually work with models trained to minimize cross-entropy loss and output softmax probabilities. In this work, we present interesting experimental results that suggest the importance of considering other loss functions and target representations, specifically, (1) training on mean-squared error and (2) representing targets as codewords generated from a random codebook. We evaluate the robustness of neural networks that implement these proposed modifications using existing attacks, showing an increase in accuracy against untargeted attacks of up to 98.7\% and a decrease of targeted attack success rates of up to 99.8\%. Our model demonstrates more robustness compared to its conventional counterpart even against attacks that are tailored to our modifications. Furthermore, we find that the parameters of our modified model have significantly smaller Lipschitz bounds, an important measure correlated with a model's sensitivity to adversarial perturbations.