SoK: Certified Robustness for Deep Neural Networks

TL;DR

This paper systematically reviews certifiably robust methods for defending deep neural networks against adversarial attacks, providing a taxonomy, benchmarking, and insights into their strengths, limitations, and future directions.

Contribution

It offers the first comprehensive benchmark and taxonomy of certifiably robust approaches, analyzing their characteristics, limitations, and connections, along with an open-source evaluation platform.

Findings

Benchmark results for 20+ approaches across datasets

Insights into strengths and limitations of certifiable methods

Identification of future research challenges

Abstract

Great advances in deep neural networks (DNNs) have led to state-of-the-art performance on a wide range of tasks. However, recent studies have shown that DNNs are vulnerable to adversarial attacks, which have brought great concerns when deploying these models to safety-critical applications such as autonomous driving. Different defense approaches have been proposed against adversarial attacks, including: a) empirical defenses, which can usually be adaptively attacked again without providing robustness certification; and b) certifiably robust approaches, which consist of robustness verification providing the lower bound of robust accuracy against any attacks under certain conditions and corresponding robust training approaches. In this paper, we systematize certifiably robust approaches and related practical and theoretical implications and findings. We also provide the first comprehensive benchmark on existing robustness verification and training approaches on different datasets. In particular, we 1) provide a taxonomy for the robustness verification and training approaches, as well as summarize the methodologies for representative algorithms, 2) reveal the characteristics, strengths, limitations, and fundamental connections among these approaches, 3) discuss current research progresses, theoretical barriers, main challenges, and future directions for certifiably robust approaches for DNNs, and 4) provide an open-sourced unified platform to evaluate 20+ representative certifiably robust approaches.