How Wrong Am I? - Studying Adversarial Examples and their Impact on Uncertainty in Gaussian Process Machine Learning Models

TL;DR

This paper explores how Gaussian Process models can detect adversarial examples through uncertainty measures, showing promising results in identifying attacks across various datasets and models.

Contribution

It introduces a Bayesian inference framework using Gaussian Processes to analyze adversarial examples and demonstrates their effectiveness in detecting attacks via uncertainty thresholds.

Findings

Uncertainty levels in Gaussian Processes reflect adversarial perturbations

Uncertainty thresholds can reject many adversarial examples

Modified attacks can bypass simple uncertainty-based defenses

Abstract

Machine learning models are vulnerable to Adversarial Examples: minor perturbations to input samples intended to deliberately cause misclassification. Current defenses against adversarial examples, especially for Deep Neural Networks (DNN), are primarily derived from empirical developments, and their security guarantees are often only justified retroactively. Many defenses therefore rely on hidden assumptions that are subsequently subverted by increasingly elaborate attacks. This is not surprising: deep learning notoriously lacks a comprehensive mathematical framework to provide meaningful guarantees. In this paper, we leverage Gaussian Processes to investigate adversarial examples in the framework of Bayesian inference. Across different models and datasets, we find deviating levels of uncertainty reflect the perturbation introduced to benign samples by state-of-the-art attacks, including novel white-box attacks on Gaussian Processes. Our experiments demonstrate that even unoptimized uncertainty thresholds already reject adversarial examples in many scenarios. Comment: Thresholds can be broken in a modified attack, which was done in arXiv:1812.02606 (The limitations of model uncertainty in adversarial settings).