InfiFusion: A Unified Framework for Enhanced Cross-Model Reasoning via LLM Fusion

TL;DR

InfiFusion is a novel framework that efficiently combines multiple domain-specific LLMs into a single model, improving performance across diverse tasks while significantly reducing training costs.

Contribution

It introduces a flexible fusion pipeline with enhanced distillation techniques and two strategies, outperforming state-of-the-art models with less computational resources.

Findings

Outperforms models like Qwen-2.5-14B-Instruct and Phi-4 on 11 benchmarks.

Reduces training cost to 160 GPU hours from millions.

Achieves superior reasoning, coding, mathematics, and instruction-following performance.

Abstract

We introduce InfiFusion, an efficient training pipeline designed to integrate multiple domain-specialized Large Language Models (LLMs) into a single pivot model, effectively harnessing the strengths of each source model. Traditional fusion methods either merge model parameters directly or rely on knowledge distillation with rigid assumptions, limiting their flexibility and efficiency. InfiFusion overcomes these limitations by enhancing Universal Logit Distillation (ULD) with Top-K selection and Logits Standardization. We propose two fusion strategies: Pairwise Fusion (InfiFusion$_p$), where each source model knowledge is distilled individually into the pivot model followed by merging and Unified Fusion (InfiFusion$_u$), where knowledge from all source models is distilled simultaneously into the pivot model. InfiFusion outperforms the state-of-the-art models, such as Qwen-2.5-14B-Instruct and Phi-4, across 11 widely applied benchmarks covering reasoning, coding, mathematics, and instruction-following tasks. Notably, InfiFusion achieves this superior performance while significantly reduces computational costs, completing full training with only 160 H800 GPU hours compared to the millions typically required for traditional LLM training.