Continual Learning for Large Language Models: A Survey

TL;DR

This survey reviews recent methods for enabling large language models to learn continuously without full retraining, categorizing techniques and discussing challenges and future directions.

Contribution

It introduces a novel multi-staged categorization scheme for continual learning techniques specific to LLMs and compares them with other adaptation strategies.

Findings

Catalogs continual learning techniques in three stages

Contrasts continual learning with retrieval-augmented methods

Identifies key challenges and future research directions

Abstract

Large language models (LLMs) are not amenable to frequent re-training, due to high training costs arising from their massive scale. However, updates are necessary to endow LLMs with new skills and keep them up-to-date with rapidly evolving human knowledge. This paper surveys recent works on continual learning for LLMs. Due to the unique nature of LLMs, we catalog continue learning techniques in a novel multi-staged categorization scheme, involving continual pretraining, instruction tuning, and alignment. We contrast continual learning for LLMs with simpler adaptation methods used in smaller models, as well as with other enhancement strategies like retrieval-augmented generation and model editing. Moreover, informed by a discussion of benchmarks and evaluation, we identify several challenges and future work directions for this crucial task.