Randomized Smoothing of All Shapes and Sizes

TL;DR

This paper develops a general theoretical framework for randomized smoothing applicable to various norms, introduces new methods for robustness guarantees, and demonstrates improved certified accuracy especially in the  norm, while also revealing fundamental limits of the approach.

Contribution

It proposes a unified theory for randomized smoothing across different norms, introduces two new methods for robustness analysis, and explores fundamental limits using Banach space theory.

Findings

Improved certified accuracy in  norm on standard datasets.

New methods for deriving robustness radii for smoothing distributions.

Fundamental limits on randomized smoothing effectiveness in high dimensions.

Abstract

Randomized smoothing is the current state-of-the-art defense with provable robustness against $\ell_2$ adversarial attacks. Many works have devised new randomized smoothing schemes for other metrics, such as $\ell_1$ or $\ell_\infty$; however, substantial effort was needed to derive such new guarantees. This begs the question: can we find a general theory for randomized smoothing? We propose a novel framework for devising and analyzing randomized smoothing schemes, and validate its effectiveness in practice. Our theoretical contributions are: (1) we show that for an appropriate notion of "optimal", the optimal smoothing distributions for any "nice" norms have level sets given by the norm's *Wulff Crystal*; (2) we propose two novel and complementary methods for deriving provably robust radii for any smoothing distribution; and, (3) we show fundamental limits to current randomized smoothing techniques via the theory of *Banach space cotypes*. By combining (1) and (2), we significantly improve the state-of-the-art certified accuracy in $\ell_1$ on standard datasets. Meanwhile, we show using (3) that with only label statistics under random input perturbations, randomized smoothing cannot achieve nontrivial certified accuracy against perturbations of $\ell_p$-norm $\Omega(\min(1, d^{\frac{1}{p} - \frac{1}{2}}))$, when the input dimension $d$ is large. We provide code in github.com/tonyduan/rs4a.