Systemizing Multiplicity: The Curious Case of Arbitrariness in Machine Learning

TL;DR

This paper systematically reviews the concept of multiplicity in machine learning, formalizing its terminology, expanding its scope, and clarifying its distinction from related notions like uncertainty, to guide responsible AI development.

Contribution

It formalizes the concept of multiplicity, broadens its definition beyond predictions, and clarifies its relationship with uncertainty and variance in machine learning models.

Findings

Provides a formal framework for multiplicity

Expands the scope of multiplicity beyond predictions

Highlights benefits and risks of multiplicity in AI

Abstract

Algorithmic modeling relies on limited information in data to extrapolate outcomes for unseen scenarios, often embedding an element of arbitrariness in its decisions. A perspective on this arbitrariness that has recently gained interest is multiplicity-the study of arbitrariness across a set of "good models", i.e., those likely to be deployed in practice. In this work, we systemize the literature on multiplicity by: (a) formalizing the terminology around model design choices and their contribution to arbitrariness, (b) expanding the definition of multiplicity to incorporate underrepresented forms beyond just predictions and explanations, (c) clarifying the distinction between multiplicity and other lenses of arbitrariness, i.e., uncertainty and variance, and (d) distilling the benefits and potential risks of multiplicity into overarching trends, situating it within the broader landscape of responsible AI. We conclude by identifying open research questions and highlighting emerging trends in this young but rapidly growing area of research.