The Effect of Prior Lipschitz Continuity on the Adversarial Robustness of Bayesian Neural Networks

TL;DR

This paper investigates how the Lipschitz continuity induced by priors in Bayesian Neural Networks influences their robustness against adversarial attacks, highlighting the importance of prior variance in enhancing safety.

Contribution

It provides an in-depth analysis of the impact of prior choices, especially prior variance, on the adversarial robustness of Bayesian Neural Networks.

Findings

Robustness is sensitive to prior variance.

Lipschitz continuity affects adversarial resistance.

Gaussian priors influence model safety.

Abstract

It is desirable, and often a necessity, for machine learning models to be robust against adversarial attacks. This is particularly true for Bayesian models, as they are well-suited for safety-critical applications, in which adversarial attacks can have catastrophic outcomes. In this work, we take a deeper look at the adversarial robustness of Bayesian Neural Networks (BNNs). In particular, we consider whether the adversarial robustness of a BNN can be increased by model choices, particularly the Lipschitz continuity induced by the prior. Conducting in-depth analysis on the case of i.i.d., zero-mean Gaussian priors and posteriors approximated via mean-field variational inference, we find evidence that adversarial robustness is indeed sensitive to the prior variance.