Merging Beyond: Streaming LLM Updates via Activation-Guided Rotations

TL;DR

This paper introduces Streaming Merging with Activation-guided Rotation (ARM), a novel iterative optimization method for updating large language models efficiently by aligning semantic subspaces, surpassing traditional fine-tuning results.

Contribution

The paper proposes ARM, a new merging strategy that treats merging as an optimization process, enabling models to improve beyond fully fine-tuned checkpoints using only early SFT data.

Findings

ARM surpasses fully converged SFT models in experiments.

ARM effectively aligns semantic subspaces to preserve geometric structure.

The method is scalable and lightweight across various model sizes and domains.

Abstract

The escalating scale of Large Language Models (LLMs) necessitates efficient adaptation techniques. Model merging has gained prominence for its efficiency and controllability. However, existing merging techniques typically serve as post-hoc refinements or focus on mitigating task interference, often failing to capture the dynamic optimization benefits of supervised fine-tuning (SFT). In this work, we propose Streaming Merging, an innovative model updating paradigm that conceptualizes merging as an iterative optimization process. Central to this paradigm is \textbf{ARM} (\textbf{A}ctivation-guided \textbf{R}otation-aware \textbf{M}erging), a strategy designed to approximate gradient descent dynamics. By treating merging coefficients as learning rates and deriving rotation vectors from activation subspaces, ARM effectively steers parameter updates along data-driven trajectories. Unlike conventional linear interpolation, ARM aligns semantic subspaces to preserve the geometric structure of high-dimensional parameter evolution. Remarkably, ARM requires only early SFT checkpoints and, through iterative merging, surpasses the fully converged SFT model. Experimental results across model scales (1.7B to 14B) and diverse domains (e.g., math, code) demonstrate that ARM can transcend converged checkpoints. Extensive experiments show that ARM provides a scalable and lightweight framework for efficient model adaptation.