ProFuser: Progressive Fusion of Large Language Models

TL;DR

ProFuser introduces a progressive fusion method for large language models that combines training and inference mode evaluations, leading to more effective model integration and improved performance in key areas.

Contribution

The paper presents a novel fusion approach that incorporates both training and inference evaluations, enabling more effective model combination.

Findings

Enhanced model performance in knowledge, reasoning, and safety.

Successful fusion of Vicuna, Llama-2, and MPT models.

ProFuser outperforms baseline fusion methods.

Abstract

While fusing the capacities and advantages of various large language models offers a pathway to construct more powerful and versatile models, a fundamental challenge is to properly select advantageous model during training. Existing fusion methods primarily focus on the training mode that uses cross entropy on ground truth in a teacher-forcing setup to measure a model's advantage, which may provide limited insight towards model advantage. In this paper, we introduce a novel approach that enhances the fusion process by incorporating both the training and inference modes. Our method evaluates model advantage not only through cross entropy during training but also by considering inference outputs, providing a more comprehensive assessment. To combine the two modes effectively, we introduce ProFuser to progressively transition from inference mode to training mode. To validate ProFuser's effectiveness, we fused three models, including Vicuna-7B-v1.5, Llama-2-7B-Chat, and MPT-7B-8K-Chat, and demonstrated the improved performance in knowledge, reasoning, and safety compared to baseline methods.