Achieving robustness in classification using optimal transport with hinge regularization

TL;DR

This paper introduces a novel optimal transport-based framework with hinge regularization for binary classification, enhancing adversarial robustness of neural networks while maintaining accuracy and providing certifiable robustness bounds.

Contribution

It proposes a new loss function integrating Lipschitz constraints via hinge regularization, ensuring robust and certifiable neural network classifiers based on optimal transport theory.

Findings

Achieves robustness guarantees without accuracy loss

Provides interpretable adversarial examples

Extends to multi-class classification

Abstract

Adversarial examples have pointed out Deep Neural Networks vulnerability to small local noise. It has been shown that constraining their Lipschitz constant should enhance robustness, but make them harder to learn with classical loss functions. We propose a new framework for binary classification, based on optimal transport, which integrates this Lipschitz constraint as a theoretical requirement. We propose to learn 1-Lipschitz networks using a new loss that is an hinge regularized version of the Kantorovich-Rubinstein dual formulation for the Wasserstein distance estimation. This loss function has a direct interpretation in terms of adversarial robustness together with certifiable robustness bound. We also prove that this hinge regularized version is still the dual formulation of an optimal transportation problem, and has a solution. We also establish several geometrical properties of this optimal solution, and extend the approach to multi-class problems. Experiments show that the proposed approach provides the expected guarantees in terms of robustness without any significant accuracy drop. The adversarial examples, on the proposed models, visibly and meaningfully change the input providing an explanation for the classification.