Effects of Exponential Gaussian Distribution on (Double Sampling) Randomized Smoothing

TL;DR

This paper investigates how exponential Gaussian distributions affect the robustness certification of randomized smoothing methods, deriving formulas and demonstrating improved certified accuracy, especially with EGG distributions, on real datasets.

Contribution

It introduces an analytic formula for ESG's certified radius, proves EGG's tighter bounds for high-dimensional cases, and empirically shows EGG's significant certification improvements.

Findings

ESG's certified radius formula converges as dimension increases.

EGG provides tighter bounds than DSRS in high dimensions.

EGG improves certified accuracy by up to 6.4% on ImageNet.

Abstract

Randomized Smoothing (RS) is currently a scalable certified defense method providing robustness certification against adversarial examples. Although significant progress has been achieved in providing defenses against $\ell_p$ adversaries, the interaction between the smoothing distribution and the robustness certification still remains vague. In this work, we comprehensively study the effect of two families of distributions, named Exponential Standard Gaussian (ESG) and Exponential General Gaussian (EGG) distributions, on Randomized Smoothing and Double Sampling Randomized Smoothing (DSRS). We derive an analytic formula for ESG's certified radius, which converges to the origin formula of RS as the dimension $d$ increases. Additionally, we prove that EGG can provide tighter constant factors than DSRS in providing $\Omega(\sqrt{d})$ lower bounds of $\ell_2$ certified radius, and thus further addresses the curse of dimensionality in RS. Our experiments on real-world datasets confirm our theoretical analysis of the ESG distributions, that they provide almost the same certification under different exponents $\eta$ for both RS and DSRS. In addition, EGG brings a significant improvement to the DSRS certification, but the mechanism can be different when the classifier properties are different. Compared to the primitive DSRS, the increase in certified accuracy provided by EGG is prominent, up to 6.4% on ImageNet.