The robust way to stack and bag: the local Lipschitz way

TL;DR

This paper leverages local Lipschitz constants to enhance adversarial robustness in neural network ensembles, proposing new architectures for bagging and stacking that outperform traditional methods on MNIST and CIFAR-10.

Contribution

It introduces a novel approach using local Lipschitz constants to design ensemble architectures that improve adversarial robustness.

Findings

Ensemble architectures based on local Lipschitz constants outperform single networks.

The proposed methods show increased robustness against FGSM and PGD attacks.

Experimental results on MNIST and CIFAR-10 validate the effectiveness of the approach.

Abstract

Recent research has established that the local Lipschitz constant of a neural network directly influences its adversarial robustness. We exploit this relationship to construct an ensemble of neural networks which not only improves the accuracy, but also provides increased adversarial robustness. The local Lipschitz constants for two different ensemble methods - bagging and stacking - are derived and the architectures best suited for ensuring adversarial robustness are deduced. The proposed ensemble architectures are tested on MNIST and CIFAR-10 datasets in the presence of white-box attacks, FGSM and PGD. The proposed architecture is found to be more robust than a) a single network and b) traditional ensemble methods.