Don't Always Pick the Highest-Performing Model: An Information Theoretic View of LLM Ensemble Selection

TL;DR

This paper introduces an information-theoretic approach to select models for LLM ensembles, demonstrating that optimal selection based on mutual information can outperform traditional methods, especially when models are correlated.

Contribution

The paper formulates ensemble selection as maximizing mutual information and proposes a greedy algorithm that accounts for model correlation, improving ensemble performance under query budgets.

Findings

The proposed method outperforms baseline ensemble selection strategies.

Model correlation causes performance saturation, which can be mitigated by mutual information-based selection.

The approach is validated on multiple NLP datasets, showing consistent improvements.

Abstract

Large language models (LLMs) are often ensembled together to improve overall reliability and robustness, but in practice models are strongly correlated. This raises a fundamental question: which models should be selected when forming an LLM ensemble? We formulate budgeted ensemble selection as maximizing the mutual information between the true label and predictions of the selected models. Furthermore, to explain why performance can saturate even with many models, we model the correlated errors of the models using Gaussian-copula and show an information-theoretic error floor for the performance of the ensemble. Motivated by these, we propose a simple greedy mutual-information selection algorithm that estimates the required information terms directly from data and iteratively builds an ensemble under a query budget. We test our approach in two question answering datasets and one binary sentiment classification dataset: MEDMCQA, MMLU, and IMDB movie reviews. Across all datasets, we observe that our method consistently outperforms strong baselines under the same query budget.