Boosting Robustness Verification of Semantic Feature Neighborhoods

TL;DR

VeeP is an active learning method that improves the scalability of robustness verification for deep neural networks against semantic feature perturbations by intelligently splitting verification tasks.

Contribution

The paper introduces VeeP, a novel active learning approach that enhances robustness verification efficiency by predicting optimal verification steps using parametric regression.

Findings

VeeP verifies 96% of neighborhoods within 29 minutes on average.

Compared to existing methods, VeeP is faster and verifies more neighborhoods within the same time limit.

VeeP successfully analyzes various semantic feature neighborhoods across multiple datasets.

Abstract

Deep neural networks have been shown to be vulnerable to adversarial attacks that perturb inputs based on semantic features. Existing robustness analyzers can reason about semantic feature neighborhoods to increase the networks' reliability. However, despite the significant progress in these techniques, they still struggle to scale to deep networks and large neighborhoods. In this work, we introduce VeeP, an active learning approach that splits the verification process into a series of smaller verification steps, each is submitted to an existing robustness analyzer. The key idea is to build on prior steps to predict the next optimal step. The optimal step is predicted by estimating the certification velocity and sensitivity via parametric regression. We evaluate VeeP on MNIST, Fashion-MNIST, CIFAR-10 and ImageNet and show that it can analyze neighborhoods of various features: brightness, contrast, hue, saturation, and lightness. We show that, on average, given a 90 minute timeout, VeeP verifies 96% of the maximally certifiable neighborhoods within 29 minutes, while existing splitting approaches verify, on average, 73% of the maximally certifiable neighborhoods within 58 minutes.