Shadow-FT: Tuning Instruct Model via Training on Paired Base Model

TL;DR

Shadow-FT is a novel fine-tuning method that leverages base models to improve instruction-tuned LLMs by directly grafting learned weight updates, leading to significant performance gains across diverse benchmarks.

Contribution

The paper introduces Shadow-FT, a simple yet effective framework for tuning instruct models by leveraging base models, avoiding additional parameters and outperforming traditional tuning methods.

Findings

Shadow-FT outperforms conventional tuning approaches across 19 benchmarks.

It can be applied to multimodal LLMs and combined with DPO.

The method significantly improves performance with no extra parameters.

Abstract

Large language models (LLMs) consistently benefit from further fine-tuning on various tasks. However, we observe that directly tuning the Instruct (i.e., instruction-tuned) models often leads to marginal improvements and even performance degeneration. Notably, paired Base models, the foundation for these Instruct variants, contain highly similar weight values (i.e., less than 2% on average for Llama 3.1 8B). The Base model tends to be a good learner yet a weak backbone without post-training. Therefore, we propose a novel Shadow-FT framework to tune the Instruct models by leveraging the corresponding Base models. The key insight is to fine-tune the Base model, and then \textit{directly} graft the learned weight updates to the Instruct model. Our proposed Shadow-FT introduces no additional parameters, is easy to implement, and significantly improves performance. We conduct extensive experiments on tuning mainstream LLMs, such as Qwen 3 and Llama 3 series, and evaluate them across 19 benchmarks covering coding, reasoning, and mathematical tasks. Experimental results demonstrate that Shadow-FT consistently outperforms conventional full-parameter and parameter-efficient tuning approaches. Further analyses indicate that Shadow-FT can be applied to multimodal large language models (MLLMs) and combined with direct preference optimization~(DPO). Codes and weights are available at \href{https://github.com/wutaiqiang/Shadow-FT}{Github}.