Boosting Certified $\ell_\infty$ Robustness with EMA Method and Ensemble Model

TL;DR

This paper enhances certified  robustness of -Lipschitz neural networks by introducing EMA training and ensemble methods, backed by theoretical analysis, to improve accuracy and stability.

Contribution

It proposes EMA and ensemble techniques specifically for -Lipschitz networks, improving certified robustness and training stability.

Findings

Significant improvement in certified accuracy with ensemble models.

Theoretical proof of robustness enhancement via ensemble methods.

Effective training process with EMA for -Lipschitz neural networks.

Abstract

The neural network with $1$-Lipschitz property based on $\ell_\infty$-dist neuron has a theoretical guarantee in certified $\ell_\infty$ robustness. However, due to the inherent difficulties in the training of the network, the certified accuracy of previous work is limited. In this paper, we propose two approaches to deal with these difficuties. Aiming at the characteristics of the training process based on $\ell_\infty$-norm neural network, we introduce the EMA method to improve the training process. Considering the randomness of the training algorithm, we propose an ensemble method based on trained base models that have the $1$-Lipschitz property and gain significant improvement in the small parameter network. Moreover, we give the theoretical analysis of the ensemble method based on the $1$-Lipschitz property on the certified robustness, which ensures the effectiveness and stability of the algorithm. Our code is available at https://github.com/Theia-4869/EMA-and-Ensemble-Lip-Networks.