Choice of PEFT Technique in Continual Learning: Prompt Tuning is Not All You Need

TL;DR

This paper critically examines the use of prompt tuning as a PEFT method in continual learning, demonstrating that alternative techniques like LoRA can significantly improve performance across various benchmarks.

Contribution

The study reveals that prompt tuning may hinder continual learning performance and shows that replacing it with LoRA enhances accuracy and competitiveness.

Findings

LoRA outperforms prompt tuning in continual learning tasks

Replacing prompt tuning with LoRA improves accuracy across benchmarks

Unexamined PEFT choices can impede progress in continual learning

Abstract

Recent Continual Learning (CL) methods have combined pretrained Transformers with prompt tuning, a parameter-efficient fine-tuning (PEFT) technique. We argue that the choice of prompt tuning in prior works was an undefended and unablated decision, which has been uncritically adopted by subsequent research, but warrants further research to understand its implications. In this paper, we conduct this research and find that the choice of prompt tuning as a PEFT method hurts the overall performance of the CL system. To illustrate this, we replace prompt tuning with LoRA in two state-of-the-art continual learning methods: Learning to Prompt and S-Prompts. These variants consistently achieve higher accuracy across a wide range of domain-incremental and class-incremental benchmarks, while being competitive in inference speed. Our work highlights a crucial argument: unexamined choices can hinder progress in the field, and rigorous ablations, such as the PEFT method, are required to drive meaningful adoption of CL techniques in real-world applications.