The Single-Multi Evolution Loop for Self-Improving Model Collaboration Systems

TL;DR

This paper introduces a novel single-multi evolution loop that enables self-improving model collaboration systems, distilling collaborative patterns into a single model to enhance efficiency and collective performance across multiple tasks.

Contribution

It proposes the single-multi evolution loop framework, allowing models to self-improve through iterative collaboration and distillation, outperforming existing evolutionary AI methods.

Findings

Models improve by 8.0% on average after distillation.

Collaboration benefits increase by 14.9% after evolution.

The method reduces computational costs by using a single distilled model.

Abstract

Model collaboration -- systems where multiple language models (LMs) collaborate -- combines the strengths of diverse models with cost in loading multiple LMs. We improve efficiency while preserving the strengths of collaboration by distilling collaborative patterns into a single model, where the model is trained on the outputs of the model collaboration system. At inference time, only the distilled model is employed: it imitates the collaboration while only incurring the cost of a single model. Furthermore, we propose the single-multi evolution loop: multiple LMs collaborate, each distills from the collaborative outputs, and these post-distillation improved LMs collaborate again, forming a collective evolution ecosystem where models evolve and self-improve by interacting with an environment of other models. Extensive experiments with 7 collaboration strategies and 15 tasks (QA, reasoning, factuality, etc.) demonstrate that: 1) individual models improve by 8.0% on average, absorbing the strengths of collaboration while reducing the cost to a single model; 2) the collaboration also benefits from the stronger and more synergistic LMs after distillation, improving over initial systems without evolution by 14.9% on average. Analysis reveals that the single-multi evolution loop outperforms various existing evolutionary AI methods, is compatible with diverse model/collaboration/distillation settings, and helps solve problems where the initial model/system struggles to.