Fast Certified Robust Training with Short Warmup
Zhouxing Shi, Yihan Wang, Huan Zhang, Jinfeng Yi, Cho-Jui Hsieh

TL;DR
This paper introduces three improvements to IBP-based certified robust training, enabling significantly faster training with shorter warmup periods while maintaining state-of-the-art robustness guarantees.
Contribution
The authors propose a new weight initialization, full Batch Normalization, and a regularization technique to improve IBP training stability and speed, reducing warmup epochs needed for certification.
Findings
Achieved 65.03% verified error on CIFAR-10 with only 160 epochs.
Achieved 82.36% verified error on TinyImageNet with only 80 epochs.
Outperformed previous methods requiring hundreds or thousands of epochs.
Abstract
Recently, bound propagation based certified robust training methods have been proposed for training neural networks with certifiable robustness guarantees. Despite that state-of-the-art (SOTA) methods including interval bound propagation (IBP) and CROWN-IBP have per-batch training complexity similar to standard neural network training, they usually use a long warmup schedule with hundreds or thousands epochs to reach SOTA performance and are thus still costly. In this paper, we identify two important issues in existing methods, namely exploded bounds at initialization, and the imbalance in ReLU activation states and improve IBP training. These two issues make certified training difficult and unstable, and thereby long warmup schedules were needed in prior works. To mitigate these issues and conduct faster certified training with shorter warmup, we propose three improvements based on IBP…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsAdversarial Robustness in Machine Learning · Advanced Neural Network Applications · Domain Adaptation and Few-Shot Learning
MethodsBatch Normalization · *Communicated@Fast*How Do I Communicate to Expedia?
