Cross-Domain Continual Learning via CLAMP

TL;DR

This paper introduces CLAMP, a novel cross-domain continual learning method that combines domain adaptation and assessor-guided learning to prevent catastrophic forgetting and improve performance in changing environments.

Contribution

CLAMP is the first approach to integrate class-aware adversarial domain adaptation with assessor-guided learning for cross-domain continual learning.

Findings

CLAMP outperforms baseline algorithms by at least 10% in various experiments.

The method effectively balances stability and plasticity in continual learning.

Theoretical analysis supports the robustness of CLAMP in complex environments.

Abstract

Artificial neural networks, celebrated for their human-like cognitive learning abilities, often encounter the well-known catastrophic forgetting (CF) problem, where the neural networks lose the proficiency in previously acquired knowledge. Despite numerous efforts to mitigate CF, it remains the significant challenge particularly in complex changing environments. This challenge is even more pronounced in cross-domain adaptation following the continual learning (CL) setting, which is a more challenging and realistic scenario that is under-explored. To this end, this article proposes a cross-domain CL approach making possible to deploy a single model in such environments without additional labelling costs. Our approach, namely continual learning approach for many processes (CLAMP), integrates a class-aware adversarial domain adaptation strategy to align a source domain and a target domain. An assessor-guided learning process is put forward to navigate the learning process of a base model assigning a set of weights to every sample controlling the influence of every sample and the interactions of each loss function in such a way to balance the stability and plasticity dilemma thus preventing the CF problem. The first assessor focuses on the negative transfer problem rejecting irrelevant samples of the source domain while the second assessor prevents noisy pseudo labels of the target domain. Both assessors are trained in the meta-learning approach using random transformation techniques and similar samples of the source domain. Theoretical analysis and extensive numerical validations demonstrate that CLAMP significantly outperforms established baseline algorithms across all experiments by at least $10\%$ margin.