Adversarial Attacks Against Uncertainty Quantification

TL;DR

This paper investigates adversarial attacks specifically targeting the uncertainty estimates of machine learning models, revealing vulnerabilities in uncertainty quantification methods across classification and segmentation tasks.

Contribution

It introduces a threat model for attacks on uncertainty quantification and develops new attack strategies, providing a comprehensive analysis of their effectiveness.

Findings

Attacks can effectively manipulate uncertainty estimates without causing misclassification.

Certain UQ techniques are more vulnerable to adversarial manipulation.

Manipulating uncertainty can undermine downstream decision processes.

Abstract

Machine-learning models can be fooled by adversarial examples, i.e., carefully-crafted input perturbations that force models to output wrong predictions. While uncertainty quantification has been recently proposed to detect adversarial inputs, under the assumption that such attacks exhibit a higher prediction uncertainty than pristine data, it has been shown that adaptive attacks specifically aimed at reducing also the uncertainty estimate can easily bypass this defense mechanism. In this work, we focus on a different adversarial scenario in which the attacker is still interested in manipulating the uncertainty estimate, but regardless of the correctness of the prediction; in particular, the goal is to undermine the use of machine-learning models when their outputs are consumed by a downstream module or by a human operator. Following such direction, we: \textit{(i)} design a threat model for attacks targeting uncertainty quantification; \textit{(ii)} devise different attack strategies on conceptually different UQ techniques spanning for both classification and semantic segmentation problems; \textit{(iii)} conduct a first complete and extensive analysis to compare the differences between some of the most employed UQ approaches under attack. Our extensive experimental analysis shows that our attacks are more effective in manipulating uncertainty quantification measures than attacks aimed to also induce misclassifications.