Cool-Fusion: Fuse Large Language Models without Training

TL;DR

Cool-Fusion presents a training-free method to fuse multiple large language models by ensemble reranking, significantly improving accuracy on benchmarks like GSM8K without the need for fine-tuning.

Contribution

It introduces Cool-Fusion, a novel approach to fuse heterogeneous LLMs without training, addressing vocabulary discrepancies through text-level ensembling and reranking.

Findings

Increases GSM8K accuracy by 17.4%

Applicable to models with different vocabularies

No training required for fusion

Abstract

We focus on the problem of fusing two or more heterogeneous large language models (LLMs) to leverage their complementary strengths. One of the challenges of model fusion is high computational load, specifically in fine-tuning or aligning vocabularies. To address this, we propose Cool-Fusion, a simple yet effective approach that fuses the knowledge of source LLMs, which does not require training. Unlike ensemble methods, Cool-Fusion is applicable to any set of source LLMs that have different vocabularies. To overcome the vocabulary discrepancies among LLMs, we ensemble LLMs on text level, allowing them to rerank the generated texts by each other with different granularities. Extensive experiments have been conducted across a variety of benchmark datasets. On GSM8K, Cool-Fusion increases accuracy from three strong source LLMs by a significant margin of 17.4\%.