Adversarial Training and Provable Robustness: A Tale of Two Objectives

TL;DR

This paper introduces a unified framework combining adversarial training and provable robustness verification, employing a novel gradient technique to improve neural network robustness with strong empirical results on MNIST and CIFAR-10.

Contribution

It presents a joint optimization approach with a new gradient method that enhances certifiable robustness, outperforming prior methods in provable adversarial defense.

Findings

Achieved 6.60% verified test error on MNIST at epsilon=0.3

Achieved 66.57% verified test error on CIFAR-10 at epsilon=8/255

Method outperforms existing approaches in provable robustness

Abstract

We propose a principled framework that combines adversarial training and provable robustness verification for training certifiably robust neural networks. We formulate the training problem as a joint optimization problem with both empirical and provable robustness objectives and develop a novel gradient-descent technique that can eliminate bias in stochastic multi-gradients. We perform both theoretical analysis on the convergence of the proposed technique and experimental comparison with state-of-the-arts. Results on MNIST and CIFAR-10 show that our method can consistently match or outperform prior approaches for provable l infinity robustness. Notably, we achieve 6.60% verified test error on MNIST at epsilon = 0.3, and 66.57% on CIFAR-10 with epsilon = 8/255.