Susceptibility of Continual Learning Against Adversarial Attacks

TL;DR

This paper reveals that current continual learning methods are highly vulnerable to adversarial attacks, with any class being easily targeted and misclassified, raising concerns about their deployment in real-world scenarios.

Contribution

The study systematically evaluates the robustness of various continual learning approaches against adversarial attacks across different scenarios.

Findings

All classes are susceptible to misclassification under attack.

Current continual learning methods lack robustness for real-world deployment.

Adversarial vulnerabilities are present in regularization, replay, and hybrid approaches.

Abstract

Recent continual learning approaches have primarily focused on mitigating catastrophic forgetting. Nevertheless, two critical areas have remained relatively unexplored: 1) evaluating the robustness of proposed methods and 2) ensuring the security of learned tasks. This paper investigates the susceptibility of continually learned tasks, including current and previously acquired tasks, to adversarial attacks. Specifically, we have observed that any class belonging to any task can be easily targeted and misclassified as the desired target class of any other task. Such susceptibility or vulnerability of learned tasks to adversarial attacks raises profound concerns regarding data integrity and privacy. To assess the robustness of continual learning approaches, we consider continual learning approaches in all three scenarios, i.e., task-incremental learning, domain-incremental learning, and class-incremental learning. In this regard, we explore the robustness of three regularization-based methods, three replay-based approaches, and one hybrid technique that combines replay and exemplar approaches. We empirically demonstrated that in any setting of continual learning, any class, whether belonging to the current or previously learned tasks, is susceptible to misclassification. Our observations identify potential limitations of continual learning approaches against adversarial attacks and highlight that current continual learning algorithms could not be suitable for deployment in real-world settings.