Measuring the Effect of Causal Disentanglement on the Adversarial Robustness of Neural Network Models

TL;DR

This study quantitatively examines how causal disentanglement in neural networks correlates with adversarial robustness, revealing a strong positive relationship and insights into confounder information's impact.

Contribution

It introduces a method to measure causal disentanglement in deterministic models and demonstrates its strong correlation with adversarial robustness across multiple datasets.

Findings

Strong correlation (r=0.820) between causal disentanglement and robustness.

Moderate negative correlation (r=-0.597) between confounder information and robustness.

Causal models show increased robustness compared to traditional neural networks.

Abstract

Causal Neural Network models have shown high levels of robustness to adversarial attacks as well as an increased capacity for generalisation tasks such as few-shot learning and rare-context classification compared to traditional Neural Networks. This robustness is argued to stem from the disentanglement of causal and confounder input signals. However, no quantitative study has yet measured the level of disentanglement achieved by these types of causal models or assessed how this relates to their adversarial robustness. Existing causal disentanglement metrics are not applicable to deterministic models trained on real-world datasets. We, therefore, utilise metrics of content/style disentanglement from the field of Computer Vision to measure different aspects of the causal disentanglement for four state-of-the-art causal Neural Network models. By re-implementing these models with a common ResNet18 architecture we are able to fairly measure their adversarial robustness on three standard image classification benchmarking datasets under seven common white-box attacks. We find a strong association (r=0.820, p=0.001) between the degree to which models decorrelate causal and confounder signals and their adversarial robustness. Additionally, we find a moderate negative association between the pixel-level information content of the confounder signal and adversarial robustness (r=-0.597, p=0.040).