Persistent Backdoor Attacks in Continual Learning

TL;DR

This paper introduces two novel persistent backdoor attacks in continual learning that remain effective over time, evade defenses, and exploit minimal adversarial influence to manipulate model outputs.

Contribution

The paper proposes two new backdoor attack methods tailored for continual learning, demonstrating their high success rates and ability to bypass existing defenses.

Findings

Both attacks achieve high success rates across various configurations.

The attacks effectively evade state-of-the-art defenses like SentiNet and I-BAU.

The methods work with static, dynamic, physical, and semantic triggers.

Abstract

Backdoor attacks pose a significant threat to neural networks, enabling adversaries to manipulate model outputs on specific inputs, often with devastating consequences, especially in critical applications. While backdoor attacks have been studied in various contexts, little attention has been given to their practicality and persistence in continual learning, particularly in understanding how the continual updates to model parameters, as new data distributions are learned and integrated, impact the effectiveness of these attacks over time. To address this gap, we introduce two persistent backdoor attacks-Blind Task Backdoor and Latent Task Backdoor-each leveraging minimal adversarial influence. Our blind task backdoor subtly alters the loss computation without direct control over the training process, while the latent task backdoor influences only a single task's training, with all other tasks trained benignly. We evaluate these attacks under various configurations, demonstrating their efficacy with static, dynamic, physical, and semantic triggers. Our results show that both attacks consistently achieve high success rates across different continual learning algorithms, while effectively evading state-of-the-art defenses, such as SentiNet and I-BAU.