Non-Parametric Probabilistic Robustness: A Conservative Metric with Optimized Perturbation Distributions

TL;DR

This paper introduces non-parametric probabilistic robustness (NPPR), a practical and conservative robustness metric for deep learning models that learns perturbation distributions directly from data, addressing limitations of fixed-distribution assumptions.

Contribution

It proposes NPPR, a novel robustness metric that does not assume predefined perturbation distributions and learns from data, enhancing robustness evaluation under uncertainty.

Findings

NPPR provides more conservative robustness estimates than traditional methods.

Experimental results on multiple datasets validate NPPR's practicality and effectiveness.

NPPR can adapt to various perturbation scenarios with high accuracy.

Abstract

Deep learning (DL) models, despite their remarkable success, remain vulnerable to small input perturbations that can cause erroneous outputs, motivating the recent proposal of probabilistic robustness (PR) as a complementary alternative to adversarial robustness (AR). However, existing PR formulations assume a fixed and known perturbation distribution, an unrealistic expectation in practice. To address this limitation, we propose non-parametric probabilistic robustness (NPPR), a more practical PR metric that does not rely on any predefined perturbation distribution. Following the non-parametric paradigm in statistical modeling, NPPR learns an optimized perturbation distribution directly from data, enabling conservative PR evaluation under distributional uncertainty. We further develop an NPPR estimator based on a Gaussian Mixture Model (GMM) with Multilayer Perceptron (MLP) heads and bicubic up-sampling, covering various input-dependent and input-independent perturbation scenarios. Theoretical analyses establish the relationships among AR, PR, and NPPR. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny ImageNet across ResNet18/50, WideResNet50 and VGG16 validate NPPR as a more practical robustness metric, showing up to 40\% more conservative (lower) PR estimates compared to assuming those common perturbation distributions used in state-of-the-arts.