Predicting the Performance of Foundation Models via Agreement-on-the-Line

TL;DR

This paper investigates how agreement-on-the-line phenomena can predict out-of-distribution performance of foundation models, emphasizing the importance of ensemble diversity and initialization randomness during light finetuning.

Contribution

It reveals that random head initialization induces agreement-on-the-line in finetuned foundation models, enabling reliable OOD performance prediction across vision and language tasks.

Findings

Random head initialization reliably induces agreement-on-the-line.

Ensembles of different pretrained foundation models show agreement-on-the-line.

Agreement-on-the-line can predict OOD performance with high precision.

Abstract

Estimating the out-of-distribution performance in regimes where labels are scarce is critical to safely deploy foundation models. Recently, it was shown that ensembles of neural networks observe the phenomena "agreement-on-the-line", which can be leveraged to reliably predict OOD performance without labels. However, in contrast to classical neural networks that are trained on in-distribution data from scratch for numerous epochs, foundation models undergo minimal finetuning from heavily pretrained weights, which may reduce the ensemble diversity needed to observe agreement-on-the-line. In our work, we demonstrate that when lightly finetuning multiple runs from a single foundation model, the choice of randomness during training (linear head initialization, data ordering, and data subsetting) can lead to drastically different levels of agreement-on-the-line in the resulting ensemble. Surprisingly, only random head initialization is able to reliably induce agreement-on-the-line in finetuned foundation models across vision and language benchmarks. Second, we demonstrate that ensembles of multiple foundation models pretrained on different datasets but finetuned on the same task can also show agreement-on-the-line. In total, by careful construction of a diverse ensemble, we can utilize agreement-on-the-line-based methods to predict the OOD performance of foundation models with high precision.