Prediction Instability in Machine Learning Ensembles

TL;DR

This paper proves that all machine learning ensembles inherently exhibit some form of prediction instability, which impacts their reliability and fairness, and discusses how to mitigate these issues with consistent models.

Contribution

The paper provides a theoretical proof of fundamental prediction instabilities in ensemble methods and analyzes their implications for model fairness and reliability.

Findings

Ensembles can ignore model agreement, change predictions without new evidence, or be manipulated by including/excluding options.

Popular tree ensembles like random forest violate fairness properties due to instability.

Using consistent models asymptotically can reduce prediction instability.

Abstract

In machine learning ensembles predictions from multiple models are aggregated. Despite widespread use and strong performance of ensembles in applied problems little is known about the mathematical properties of aggregating models and associated consequences for safe, explainable use of such models. In this paper we prove a theorem that shows that any ensemble will exhibit at least one of the following forms of prediction instability. It will either ignore agreement among all underlying models, change its mind when none of the underlying models have done so, or be manipulable through inclusion or exclusion of options it would never actually predict. As a consequence, ensemble aggregation procedures will always need to balance the benefits of information use against the risk of these prediction instabilities. This analysis also sheds light on what specific forms of prediction instability to expect from particular ensemble algorithms; for example popular tree ensembles like random forest, or xgboost will violate basic, intuitive fairness properties. Finally, we show that this can be ameliorated by using consistent models in asymptotic conditions.