Mitigating Adversarial Vulnerability through Causal Parameter Estimation by Adversarial Double Machine Learning

TL;DR

This paper introduces a causal inference method called Adversarial Double Machine Learning (ADML) to quantify and mitigate adversarial vulnerabilities in neural networks, improving robustness across various architectures.

Contribution

The paper proposes a novel causal approach, ADML, that directly estimates adversarial vulnerability and enhances robustness beyond existing defense methods.

Findings

ADML significantly improves adversarial robustness across CNN and Transformer models.

It effectively quantifies the degree of vulnerability for different network predictions.

The approach reduces the impact of adversarial perturbations, leading to more resilient models.

Abstract

Adversarial examples derived from deliberately crafted perturbations on visual inputs can easily harm decision process of deep neural networks. To prevent potential threats, various adversarial training-based defense methods have grown rapidly and become a de facto standard approach for robustness. Despite recent competitive achievements, we observe that adversarial vulnerability varies across targets and certain vulnerabilities remain prevalent. Intriguingly, such peculiar phenomenon cannot be relieved even with deeper architectures and advanced defense methods. To address this issue, in this paper, we introduce a causal approach called Adversarial Double Machine Learning (ADML), which allows us to quantify the degree of adversarial vulnerability for network predictions and capture the effect of treatments on outcome of interests. ADML can directly estimate causal parameter of adversarial perturbations per se and mitigate negative effects that can potentially damage robustness, bridging a causal perspective into the adversarial vulnerability. Through extensive experiments on various CNN and Transformer architectures, we corroborate that ADML improves adversarial robustness with large margins and relieve the empirical observation.