Confidence-aware Training of Smoothed Classifiers for Certified Robustness

TL;DR

This paper introduces a simple training approach that uses a noise-based accuracy proxy to selectively enhance the robustness of smoothed classifiers, leading to improved certified robustness against adversarial attacks and corruptions.

Contribution

It proposes a novel sample-wise robustness control method leveraging Gaussian noise accuracy as a proxy, improving certified robustness of smoothed classifiers.

Findings

Consistently improves certified robustness over state-of-the-art methods.

Enhancements persist across different robustness notions, including corruptions.

Simple method yields significant robustness gains without complex procedures.

Abstract

Any classifier can be "smoothed out" under Gaussian noise to build a new classifier that is provably robust to $\ell_2$-adversarial perturbations, viz., by averaging its predictions over the noise via randomized smoothing. Under the smoothed classifiers, the fundamental trade-off between accuracy and (adversarial) robustness has been well evidenced in the literature: i.e., increasing the robustness of a classifier for an input can be at the expense of decreased accuracy for some other inputs. In this paper, we propose a simple training method leveraging this trade-off to obtain robust smoothed classifiers, in particular, through a sample-wise control of robustness over the training samples. We make this control feasible by using "accuracy under Gaussian noise" as an easy-to-compute proxy of adversarial robustness for an input. Specifically, we differentiate the training objective depending on this proxy to filter out samples that are unlikely to benefit from the worst-case (adversarial) objective. Our experiments show that the proposed method, despite its simplicity, consistently exhibits improved certified robustness upon state-of-the-art training methods. Somewhat surprisingly, we find these improvements persist even for other notions of robustness, e.g., to various types of common corruptions.