TRGP: Trust Region Gradient Projection for Continual Learning

TL;DR

TRGP introduces a trust region-based gradient projection method for continual learning that selectively reuses old task knowledge, improving transfer and reducing forgetting.

Contribution

It proposes a novel trust region concept for task selection and a scaled weight projection for efficient knowledge reuse in continual learning.

Findings

Significantly outperforms state-of-the-art methods

Effectively balances knowledge transfer and forgetting

Demonstrates robustness across multiple tasks

Abstract

Catastrophic forgetting is one of the major challenges in continual learning. To address this issue, some existing methods put restrictive constraints on the optimization space of the new task for minimizing the interference to old tasks. However, this may lead to unsatisfactory performance for the new task, especially when the new task is strongly correlated with old tasks. To tackle this challenge, we propose Trust Region Gradient Projection (TRGP) for continual learning to facilitate the forward knowledge transfer based on an efficient characterization of task correlation. Particularly, we introduce a notion of `trust region' to select the most related old tasks for the new task in a layer-wise and single-shot manner, using the norm of gradient projection onto the subspace spanned by task inputs. Then, a scaled weight projection is proposed to cleverly reuse the frozen weights of the selected old tasks in the trust region through a layer-wise scaling matrix. By jointly optimizing the scaling matrices and the model, where the model is updated along the directions orthogonal to the subspaces of old tasks, TRGP can effectively prompt knowledge transfer without forgetting. Extensive experiments show that our approach achieves significant improvement over related state-of-the-art methods.