Joint Flashback Adaptation for Forgetting-Resistant Instruction Tuning

TL;DR

The paper introduces Joint Flashback Adaptation, a method that uses limited prompts from old tasks to improve incremental learning in large language models, reducing forgetting and enhancing generalization without access to original data.

Contribution

It proposes a novel approach that leverages flashbacks and latent task interpolation for task-agnostic, data-efficient continual learning in large language models.

Findings

Outperforms existing methods in 1000+ instruction tasks.

Reduces catastrophic forgetting effectively.

Enhances generalization to new tasks.

Abstract

Large language models have achieved remarkable success in various tasks. However, it is challenging for them to learn new tasks incrementally due to catastrophic forgetting. Existing approaches rely on experience replay, optimization constraints, or task differentiation, which encounter strict limitations in real-world scenarios. To address these issues, we propose Joint Flashback Adaptation. We first introduce flashbacks -- a limited number of prompts from old tasks -- when adapting to new tasks and constrain the deviations of the model outputs compared to the original one. We then interpolate latent tasks between flashbacks and new tasks to enable jointly learning relevant latent tasks, new tasks, and flashbacks, alleviating data sparsity in flashbacks and facilitating knowledge sharing for smooth adaptation. Our method requires only a limited number of flashbacks without access to the replay data and is task-agnostic. We conduct extensive experiments on state-of-the-art large language models across 1000+ instruction-following tasks, arithmetic reasoning tasks, and general reasoning tasks. The results demonstrate the superior performance of our method in improving generalization on new tasks and reducing forgetting in old tasks.