CTBENCH: A Library and Benchmark for Certified Training
Yuhao Mao, Stefan Balauca, Martin Vechev

TL;DR
CTBench is a unified library and benchmark for certifiably robust neural network training, enabling fair comparison, systematic tuning, and new insights into current methods and their limitations.
Contribution
The paper introduces CTBench, a comprehensive benchmark and library for certified training, standardizing evaluation and revealing new insights and state-of-the-art performance.
Findings
Most algorithms outperform previous literature significantly.
Fair training and hyperparameter tuning reduce claimed advantages of recent algorithms.
Certified models exhibit less fragmented loss surfaces, shared mistakes, and sparser activations.
Abstract
Training certifiably robust neural networks is an important but challenging task. While many algorithms for (deterministic) certified training have been proposed, they are often evaluated on different training schedules, certification methods, and systematically under-tuned hyperparameters, making it difficult to compare their performance. To address this challenge, we introduce CTBench, a unified library and a high-quality benchmark for certified training that evaluates all algorithms under fair settings and systematically tuned hyperparameters. We show that (1) almost all algorithms in CTBench surpass the corresponding reported performance in literature in the magnitude of algorithmic improvements, thus establishing new state-of-the-art, and (2) the claimed advantage of recent algorithms drops significantly when we enhance the outdated baselines with a fair training schedule, a fair…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsAdversarial Robustness in Machine Learning · Advanced Neural Network Applications · Machine Learning and Data Classification
MethodsLib
