Key-Value Pair-Free Continual Learner via Task-Specific Prompt-Prototype

TL;DR

This paper introduces a key-value pair-free continual learning method using task-specific prompts and prototypes, improving scalability and reducing interference in multi-task learning scenarios.

Contribution

The proposed approach eliminates key-value pairs in prompt-based continual learning, using task-specific prompts and prototypes for better feature learning and stability.

Findings

Outperforms traditional key-value prompt methods on multiple datasets

Reduces inter-task interference and improves scalability

Enhances stability with regularization during prompt initialization

Abstract

Continual learning aims to enable models to acquire new knowledge while retaining previously learned information. Prompt-based methods have shown remarkable performance in this domain; however, they typically rely on key-value pairing, which can introduce inter-task interference and hinder scalability. To overcome these limitations, we propose a novel approach employing task-specific Prompt-Prototype (ProP), thereby eliminating the need for key-value pairs. In our method, task-specific prompts facilitate more effective feature learning for the current task, while corresponding prototypes capture the representative features of the input. During inference, predictions are generated by binding each task-specific prompt with its associated prototype. Additionally, we introduce regularization constraints during prompt initialization to penalize excessively large values, thereby enhancing stability. Experiments on several widely used datasets demonstrate the effectiveness of the proposed method. In contrast to mainstream prompt-based approaches, our framework removes the dependency on key-value pairs, offering a fresh perspective for future continual learning research.