On the Convergence of Certified Robust Training with Interval Bound Propagation

TL;DR

This paper provides a theoretical analysis demonstrating that under certain conditions, Interval Bound Propagation (IBP) training for neural networks converges linearly to zero robust training error, enhancing understanding of its effectiveness.

Contribution

It offers the first convergence proof for IBP training on overparameterized neural networks with logistic loss, under specific assumptions.

Findings

IBP training converges linearly to zero robust error under certain conditions.

Convergence holds for overparameterized two-layer ReLU networks with logistic loss.

High probability guarantees are provided for the convergence results.

Abstract

Interval Bound Propagation (IBP) is so far the base of state-of-the-art methods for training neural networks with certifiable robustness guarantees when potential adversarial perturbations present, while the convergence of IBP training remains unknown in existing literature. In this paper, we present a theoretical analysis on the convergence of IBP training. With an overparameterized assumption, we analyze the convergence of IBP robust training. We show that when using IBP training to train a randomly initialized two-layer ReLU neural network with logistic loss, gradient descent can linearly converge to zero robust training error with a high probability if we have sufficiently small perturbation radius and large network width.