Robust Learning with Jacobian Regularization

TL;DR

This paper introduces a Jacobian regularization technique that enhances neural network robustness against input perturbations, including adversarial attacks, while maintaining good generalization on clean data.

Contribution

It develops an efficient Jacobian regularization method that increases classification margins and improves robustness without sacrificing accuracy on unperturbed data.

Findings

Significant robustness improvements against adversarial attacks.

Enhanced classification margins through Jacobian regularization.

Maintained generalization performance on clean data.

Abstract

Design of reliable systems must guarantee stability against input perturbations. In machine learning, such guarantee entails preventing overfitting and ensuring robustness of models against corruption of input data. In order to maximize stability, we analyze and develop a computationally efficient implementation of Jacobian regularization that increases classification margins of neural networks. The stabilizing effect of the Jacobian regularizer leads to significant improvements in robustness, as measured against both random and adversarial input perturbations, without severely degrading generalization properties on clean data.