Get Global Guarantees: On the Probabilistic Nature of Perturbation Robustness

TL;DR

This paper introduces a new probabilistic robustness metric called tower robustness, which improves the efficiency and rigor of pre-deployment robustness assessments for safety-critical deep learning models.

Contribution

It proposes a novel, practical metric for probabilistic robustness based on hypothesis testing, addressing limitations of existing methods.

Findings

Tower robustness enables more rigorous robustness evaluation.

The proposed method reduces computational costs compared to existing approaches.

Extensive evaluation demonstrates the effectiveness of tower robustness.

Abstract

In safety-critical deep learning applications, robustness measures the ability of neural models that handle imperceptible perturbations in input data, which may lead to potential safety hazards. Existing pre-deployment robustness assessment methods typically suffer from significant trade-offs between computational cost and measurement precision, limiting their practical utility. To address these limitations, this paper conducts a comprehensive comparative analysis of existing robustness definitions and associated assessment methodologies. We propose tower robustness to evaluate robustness, which is a novel, practical metric based on hypothesis testing to quantitatively evaluate probabilistic robustness, enabling more rigorous and efficient pre-deployment assessments. Our extensive comparative evaluation illustrates the advantages and applicability of our proposed approach, thereby advancing the systematic understanding and enhancement of model robustness in safety-critical deep learning applications.