Consistency Regularization for Certified Robustness of Smoothed Classifiers

TL;DR

This paper introduces a regularization technique that enhances the certified robustness of smoothed classifiers by enforcing prediction consistency under noise, leading to improved robustness with less computational effort.

Contribution

It proposes a novel regularization method that controls the accuracy-robustness trade-off without approximating the smoothed classifier, improving certified robustness efficiently.

Findings

Significant robustness improvements across neural architectures and datasets.

Achieves comparable or better results than state-of-the-art methods.

Reduces training costs and hyperparameter tuning complexity.

Abstract

A recent technique of randomized smoothing has shown that the worst-case (adversarial) $\ell_2$-robustness can be transformed into the average-case Gaussian-robustness by "smoothing" a classifier, i.e., by considering the averaged prediction over Gaussian noise. In this paradigm, one should rethink the notion of adversarial robustness in terms of generalization ability of a classifier under noisy observations. We found that the trade-off between accuracy and certified robustness of smoothed classifiers can be greatly controlled by simply regularizing the prediction consistency over noise. This relationship allows us to design a robust training objective without approximating a non-existing smoothed classifier, e.g., via soft smoothing. Our experiments under various deep neural network architectures and datasets show that the "certified" $\ell_2$-robustness can be dramatically improved with the proposed regularization, even achieving better or comparable results to the state-of-the-art approaches with significantly less training costs and hyperparameters.