Adversarial Robustness via Fisher-Rao Regularization

TL;DR

This paper introduces FIRE, a Fisher-Rao regularization method based on information geometry, to improve adversarial robustness of neural networks, achieving better accuracy and robustness with reduced training time.

Contribution

The paper proposes a novel Fisher-Rao regularization for adversarial defense, deriving explicit geometric properties and demonstrating superior performance over existing methods.

Findings

FIRE reaches all Pareto-optimal points in accuracy-robustness trade-off.

Empirical results show up to 1% improvement in clean and robust accuracy.

Training time is reduced by 20% compared to state-of-the-art methods.

Abstract

Adversarial robustness has become a topic of growing interest in machine learning since it was observed that neural networks tend to be brittle. We propose an information-geometric formulation of adversarial defense and introduce FIRE, a new Fisher-Rao regularization for the categorical cross-entropy loss, which is based on the geodesic distance between the softmax outputs corresponding to natural and perturbed input features. Based on the information-geometric properties of the class of softmax distributions, we derive an explicit characterization of the Fisher-Rao Distance (FRD) for the binary and multiclass cases, and draw some interesting properties as well as connections with standard regularization metrics. Furthermore, for a simple linear and Gaussian model, we show that all Pareto-optimal points in the accuracy-robustness region can be reached by FIRE while other state-of-the-art methods fail. Empirically, we evaluate the performance of various classifiers trained with the proposed loss on standard datasets, showing up to a simultaneous 1\% of improvement in terms of clean and robust performances while reducing the training time by 20\% over the best-performing methods.