Merging in a Bottle: Differentiable Adaptive Merging (DAM) and the Path from Averaging to Automation

TL;DR

This paper introduces Differentiable Adaptive Merging (DAM), a novel, efficient method for model merging that optimizes integration through scaling coefficients, offering a practical alternative to more complex approaches.

Contribution

The paper presents DAM, a differentiable, adaptive merging technique that reduces computational costs and improves model integration, compared to existing methods like evolutionary strategies and hyperparameter tuning.

Findings

Simple averaging methods perform well with similar models.

DAM outperforms some existing merging techniques in efficiency.

Model merging techniques have varying strengths depending on model similarity.

Abstract

By merging models, AI systems can combine the distinct strengths of separate language models, achieving a balance between multiple capabilities without requiring substantial retraining. However, the integration process can be intricate due to differences in training methods and fine-tuning, typically necessitating specialized knowledge and repeated refinement. This paper explores model merging techniques across a spectrum of complexity, examining where automated methods like evolutionary strategies stand compared to hyperparameter-driven approaches such as DARE, TIES-Merging and simpler methods like Model Soups. In addition, we introduce Differentiable Adaptive Merging (DAM), an efficient, adaptive merging approach as an alternative to evolutionary merging that optimizes model integration through scaling coefficients, minimizing computational demands. Our findings reveal that even simple averaging methods, like Model Soups, perform competitively when model similarity is high, underscoring each technique's unique strengths and limitations. We open-sourced DAM, including the implementation code and experiment pipeline, on GitHub: https://github.com/arcee-ai/DAM.