MUSCLE: A Model Update Strategy for Compatible LLM Evolution

TL;DR

This paper introduces MUSCLE, a strategy for updating large language models that minimizes performance regressions on individual instances, thereby maintaining user trust and model consistency across updates.

Contribution

The paper proposes a novel training method involving a compatibility adapter to reduce instance-level regressions during LLM updates, improving model stability.

Findings

Negative flips reduced by up to 40% with the proposed method

Model update regressions occur even with identical training procedures

Evaluation metrics tailored for generative and discriminative tasks

Abstract

Large Language Models (LLMs) are regularly updated to enhance performance, typically through changes in data or architecture. Within the update process, developers often prioritize improving overall performance metrics, paying less attention to maintaining compatibility with earlier model versions. Instance-level degradation (instance regression) of performance from one model version to the next can interfere with a user's mental model of the capabilities of a particular language model. Users having to adapt their mental model with every update can lead to dissatisfaction, especially when the new model has degraded compared to a prior version for a known use case (model update regression). We find that when pretrained LLM base models are updated, fine-tuned user-facing downstream task adapters experience negative flips -- previously correct instances are now predicted incorrectly. We observe model update regression between different model versions on a diverse set of tasks and models, even when the downstream task training procedures remain identical. We argue for the importance of maintaining model update compatibility during updates, and present evaluation metrics designed specifically for generative tasks, while also being applicable to discriminative tasks. We propose a training strategy to minimize the extent of instance regression in model updates, involving training of a compatibility adapter that can enhance task fine-tuned language models. We show negative flips reduce by up to 40% e.g. when updating Llama 1 to Llama 2 with our proposed method.