Exploring Cross-Stage Adversarial Transferability in Class-Incremental Continual Learning

TL;DR

This paper investigates the vulnerability of class-incremental continual learning models to stage-transferred adversarial attacks, revealing high susceptibility and limited defense effectiveness, which raises security concerns.

Contribution

It is the first study to analyze cross-stage adversarial transferability in continual learning, highlighting security vulnerabilities and explaining underlying causes.

Findings

Continual learning models are highly susceptible to stage-transferred attacks.

Existing adversarial defenses are ineffective against these attacks.

Model similarity and robustness degradation explain transferability.

Abstract

Class-incremental continual learning addresses catastrophic forgetting by enabling classification models to preserve knowledge of previously learned classes while acquiring new ones. However, the vulnerability of the models against adversarial attacks during this process has not been investigated sufficiently. In this paper, we present the first exploration of vulnerability to stage-transferred attacks, i.e., an adversarial example generated using the model in an earlier stage is used to attack the model in a later stage. Our findings reveal that continual learning methods are highly susceptible to these attacks, raising a serious security issue. We explain this phenomenon through model similarity between stages and gradual robustness degradation. Additionally, we find that existing adversarial training-based defense methods are not sufficiently effective to stage-transferred attacks. Codes are available at https://github.com/mcml-official/CSAT.