Evolutionary Optimization of Model Merging Recipes

TL;DR

This paper introduces an evolutionary method for automatically merging diverse open-source models, resulting in state-of-the-art Japanese language and vision models without extensive training, advancing automated model composition.

Contribution

It presents a novel evolutionary approach for model merging that operates in parameter and data flow space, enabling cross-domain model creation without extensive training.

Findings

Japanese Math LLM achieved state-of-the-art performance on benchmarks.

Japanese VLM outperformed previous models in culture-specific tasks.

Method enables efficient, automated model merging across domains.

Abstract

Large language models (LLMs) have become increasingly capable, but their development often requires substantial computational resources. While model merging has emerged as a cost-effective promising approach for creating new models by combining existing ones, it currently relies on human intuition and domain knowledge, limiting its potential. Here, we propose an evolutionary approach that overcomes this limitation by automatically discovering effective combinations of diverse open-source models, harnessing their collective intelligence without requiring extensive additional training data or compute. Our approach operates in both parameter space and data flow space, allowing for optimization beyond just the weights of the individual models. This approach even facilitates cross-domain merging, generating models like a Japanese LLM with Math reasoning capabilities. Surprisingly, our Japanese Math LLM achieved state-of-the-art performance on a variety of established Japanese LLM benchmarks, even surpassing models with significantly more parameters, despite not being explicitly trained for such tasks. Furthermore, a culturally-aware Japanese VLM generated through our approach demonstrates its effectiveness in describing Japanese culture-specific content, outperforming previous Japanese VLMs. This work not only contributes new state-of-the-art models back to the open-source community, but also introduces a new paradigm for automated model composition, paving the way for exploring alternative, efficient approaches to foundation model development.