Knowledge is a Region in Weight Space for Fine-tuned Language Models

TL;DR

This paper explores the structure of weight space in fine-tuned language models, revealing that high-performing models occupy well-defined regions, and that traversing these regions can produce models with improved or comparable performance.

Contribution

It introduces the concept of a 'region' in weight space where fine-tuned models reside, and proposes a method for selecting better models based on this insight.

Findings

Fine-tuned models form tight clusters in weight space.

Traversing regions between models can yield better performance.

Starting from the region center improves fine-tuning efficiency.

Abstract

Research on neural networks has focused on understanding a single model trained on a single dataset. However, relatively little is known about the relationships between different models, particularly those trained or tested on different datasets. We address this by studying how the weight space and the underlying loss landscape of different models are interconnected. Specifically, we demonstrate that finetuned models that were optimized for high performance, reside in well-defined regions in weight space, and vice versa -- that any model that resides anywhere in those regions also exhibits high performance. Notably, we show that language models that have been finetuned on the same dataset form a tight cluster in the weight space, while models finetuned on different datasets from the same underlying task form a looser cluster. Moreover, traversing around the region between the models leads to new models that perform comparably or even better than models obtained via finetuning, even on tasks that the original models were not finetuned on. Our findings provide insight into the relationships between models, demonstrating that a model positioned between two similar models can acquire the knowledge of both. We leverage this and design a method for selecting a better model for efficient finetuning. Specifically, we show that starting from the center of the region is as effective, if not more, than using the pretrained model in 11 out of 12 datasets, resulting in an average accuracy improvement of 3.06.