Large Continual Instruction Assistant
Jingyang Qiao, Zhizhong Zhang, Xin Tan, Yanyun Qu, Shouhong Ding, Yuan Xie

TL;DR
This paper introduces a novel continual instruction tuning framework that balances plasticity and stability using an adaptive coefficient, significantly reducing forgetting and improving performance on multiple benchmarks.
Contribution
It proposes a general framework with an adaptive balance mechanism based on Taylor expansion, addressing stability-plasticity trade-off in continual instruction tuning.
Findings
Enhanced anti-forgetting capabilities demonstrated.
Significant performance improvements on multiple benchmarks.
Adaptive balance weight effectively manages knowledge interference.
Abstract
Continual Instruction Tuning (CIT) is adopted to continually instruct Large Models to follow human intent data by data. It is observed that existing gradient update would heavily destroy the performance on previous datasets during CIT process. Instead, Exponential Moving Average (EMA), owns the ability to trace previous parameters, which can aid in decreasing forgetting. Nonetheless, its stable balance weight fails to deal with the ever-changing datasets, leading to the out-of-balance between plasticity and stability. In this paper, we propose a general continual instruction tuning framework to address the challenge. Starting from the trade-off prerequisite and EMA update, we propose the plasticity and stability ideal condition. Based on Taylor expansion in the loss function, we find the optimal balance weight can be automatically determined by the gradients and learned parameters.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsNatural Language Processing Techniques · Topic Modeling · Speech and dialogue systems
