Certified vs. Empirical Adversarial Robust-ness via Hybrid Convolutions with Attention Stochasticity

TL;DR

HyCAS introduces a hybrid convolutional architecture with stochastic attention mechanisms that enhances both certified and empirical adversarial robustness across multiple imaging benchmarks.

Contribution

It unifies deterministic and randomized methods to improve adversarial robustness with formal certification and empirical performance, a novel combination in this context.

Findings

Surpasses prior defenses with up to 7.3% certified accuracy boost.

Improves empirical robustness by up to 3.1%.

Maintains strong clean accuracy across benchmarks.

Abstract

We introduce Hybrid Convolutions with Attention Stochasticity (HyCAS), an adversarial defense that narrows the long-standing gap between provable robustness under L2 certificates and empirical robustness against strong L attacks, while preserving strong generalization across diverse imaging benchmarks. HyCAS unifies deterministic and randomized principles by coupling 1-Lipschitz, spectrally normalized convolutions with two stochastic components, spectral normalized random, projection filters and a randomized attention-noise mechanism, to realize a randomized defense. Injecting smoothing randomness inside the architecture yields an overall <= 2-Lipschitz network with formal certificates. Exten-sive experiments on diverse imaging benchmarks, including CIFAR-10/100, ImageNet-1k, NIH Chest X-ray, HAM10000, show that HyCAS surpasses prior leading certified and empirical defenses, boosting certified accuracy by up to 7.3% (on NIH Chest X-ray) and empirical robustness by up to 3.1% (on HAM10000), without sacrificing clean accuracy. These results show that a randomized Lipschitz constrained architecture can simultaneously improve both certified L2 and empirical L adversarial robustness, thereby supporting safer deployment of deep models in high-stakes applications. Code: https://github.com/misti1203/HyCAS