Revitalizing the Beginning: Avoiding Storage Dependency for Model Merging in Continual Learning

TL;DR

This paper introduces Trajectory Regularized Merging (TRM), a novel framework for model merging in continual learning that enhances stability and optimization without relying on storage of previous data.

Contribution

The paper proposes TRM, a new merging method reformulating the process as an optimization in an augmented trajectory space, improving continual learning performance.

Findings

TRM achieves state-of-the-art results on multiple benchmarks.

Current methods tend to amplify task-specific errors and suffer from vanishing gradients.

TRM effectively preserves historical stability and re-activates optimization dynamics.

Abstract

Model merging provides a compelling paradigm for integrating specialized expertise into a unified multi-task model, a goal that aligns naturally with the sequential knowledge acquisition in continual learning (CL). However, the requirement for preserving diverse forms of previous knowledge conflicts with the storage limitations inherent to CL. In this paper, we systematically analyze existing model merging methods under the constraints of CL. We find that current methods prioritize global alignment, which often leads to the accumulation and amplification of task-specific errors within the continuous data stream; and the vanishing gradients at the onset of subsequent tasks frequently cause optimization to stagnate. These leave the merged model in a suboptimal state at the beginning of the next training phase. To address these challenges, we propose Trajectory Regularized Merging (TRM), a framework that reformulates the merging phase as an optimization process within an augmented trajectory subspace. Our framework integrates three synergistic objectives including task alignment, prediction consistency, and gradient responsiveness to concurrently preserve merged model's historical stability and re-activate optimization dynamics. Extensive experimental results demonstrate that our method achieves state-of-the-art performance across multiple benchmarks.