SHIELD: Secure Hypernetworks for Incremental Expansion Learning Defense

TL;DR

SHIELD is a novel framework combining hypernetworks and interval bound propagation to enable certifiably robust continual learning across tasks without replay buffers, achieving state-of-the-art accuracy under adversarial attacks.

Contribution

The paper introduces SHIELD, a hypernetwork-based continual learning method with Interval MixUp for certified robustness, eliminating the need for replay buffers and enabling scalable, robust incremental learning.

Findings

Outperforms existing methods under PGD and AutoAttack.

Achieves state-of-the-art average accuracy in robust continual learning.

Provides certified robustness guarantees with interval arithmetic.

Abstract

Continual learning under adversarial conditions remains an open problem, as existing methods often compromise either robustness, scalability, or both. We propose a novel framework that integrates Interval Bound Propagation (IBP) with a hypernetwork-based architecture to enable certifiably robust continual learning across sequential tasks. Our method, SHIELD, generates task-specific model parameters via a shared hypernetwork conditioned solely on compact task embeddings, eliminating the need for replay buffers or full model copies and enabling efficient over time. To further enhance robustness, we introduce Interval MixUp, a novel training strategy that blends virtual examples represented as $\ell_{\infty}$ balls centered around MixUp points. Leveraging interval arithmetic, this technique guarantees certified robustness while mitigating the wrapping effect, resulting in smoother decision boundaries. We evaluate SHIELD under strong white-box adversarial attacks, including PGD and AutoAttack, across multiple benchmarks. It consistently outperforms existing robust continual learning methods, achieving state-of-the-art average accuracy while maintaining both scalability and certification. These results represent a significant step toward practical and theoretically grounded continual learning in adversarial settings.