Learning to Prompt for Continual Learning

TL;DR

This paper introduces a novel continual learning approach called Learning to Prompt (L2P), which uses dynamic prompts to adapt a pre-trained model to new tasks without relying on task labels or rehearsal buffers.

Contribution

L2P is a new paradigm that trains a succinct memory of prompts to manage task-invariant and task-specific knowledge, outperforming prior methods without needing task identity at test time.

Findings

L2P outperforms state-of-the-art methods on image classification benchmarks.

L2P performs competitively against rehearsal-based methods without using a rehearsal buffer.

L2P is effective in task-agnostic continual learning scenarios.

Abstract

The mainstream paradigm behind continual learning has been to adapt the model parameters to non-stationary data distributions, where catastrophic forgetting is the central challenge. Typical methods rely on a rehearsal buffer or known task identity at test time to retrieve learned knowledge and address forgetting, while this work presents a new paradigm for continual learning that aims to train a more succinct memory system without accessing task identity at test time. Our method learns to dynamically prompt (L2P) a pre-trained model to learn tasks sequentially under different task transitions. In our proposed framework, prompts are small learnable parameters, which are maintained in a memory space. The objective is to optimize prompts to instruct the model prediction and explicitly manage task-invariant and task-specific knowledge while maintaining model plasticity. We conduct comprehensive experiments under popular image classification benchmarks with different challenging continual learning settings, where L2P consistently outperforms prior state-of-the-art methods. Surprisingly, L2P achieves competitive results against rehearsal-based methods even without a rehearsal buffer and is directly applicable to challenging task-agnostic continual learning. Source code is available at https://github.com/google-research/l2p.