Incremental Randomized Smoothing Certification
Shubham Ugare, Tarun Suresh, Debangshu Banerjee, Gagandeep Singh, Sasa, Misailovic

TL;DR
This paper introduces IRS, an incremental certification method for randomized smoothing that reuses previous guarantees to efficiently certify modified neural networks, significantly reducing computational costs.
Contribution
IRS is the first approach to reuse certification guarantees for modified smoothed models, enabling faster certification with minimal samples.
Findings
Up to 3x speedup in certification time
Maintains strong robustness guarantees
Effective for models after modifications like quantization or pruning
Abstract
Randomized smoothing-based certification is an effective approach for obtaining robustness certificates of deep neural networks (DNNs) against adversarial attacks. This method constructs a smoothed DNN model and certifies its robustness through statistical sampling, but it is computationally expensive, especially when certifying with a large number of samples. Furthermore, when the smoothed model is modified (e.g., quantized or pruned), certification guarantees may not hold for the modified DNN, and recertifying from scratch can be prohibitively expensive. We present the first approach for incremental robustness certification for randomized smoothing, IRS. We show how to reuse the certification guarantees for the original smoothed model to certify an approximated model with very few samples. IRS significantly reduces the computational cost of certifying modified DNNs while maintaining…
Peer Reviews
Decision·ICLR 2024 poster
- The paper is well-written and easy to follow. The motivation is clear and important. - The methodology is sound and it is friendly to read although it can be formally expressed with more complicated notations. - The experiment is extensive and validates the effectiveness and efficiency of the method.
- Insight 1 in Section 3.1 is not very convincing in the sense of a single setting of n=1k and $\sigma=1$, where it usually costs 10k-100k samples for Monte Carlo sampling in estimation. More examples can be given to show $\zeta$ is small. - For insight 2 and Figure 2, although the needed samples are much less compared to 0.5, it still needs 41.5k and there is no significant reduction compred to naive Monte Carlo randomized smoothing (10k-100k). A better way is to use an example of current estim
1. This work proposed a first incremental approach for randomized smoothing to certify a similar (compressed) version of the original neural network with improved efficiency by re-using the certification results. 2. The experiments results seem to be promising.
1. Demanding prerequisite: I am not sure how likely IRS algorithm is applicable in practice. It seems like IRS will require many prerequisite. For example, IRS needs to know the certification cache from the original neural network, which makes the requirement more demanding. If there is no such information, regular RS is still needed. As another requirement, IRS needs the modified network to be a good approximation of the original neural network. Otherwise, the accuracy might be reduced per theo
- The idea of reusing the observations for calculating the certificate for $g$ to calculate the certificate of $g_p$ is novel and interesting. - The authors also use a great insight that it is more efficient to estimate binomial parameters at extreme ends than near the middle. - The paper is well-written and easy to understand.
- The practical usefulness of the proposed method is not clear. As randomized smoothing produces certificates at inference time, in order to calculate the certificate around a given point in this approach, the edge device would need access to both the original as well as the modified neural network models, which is not feasible.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdversarial Robustness in Machine Learning · Advanced Neural Network Applications · Anomaly Detection Techniques and Applications
MethodsRandomized Smoothing
