What is it for a Machine Learning Model to Have a Capability?

TL;DR

This paper explores what it means for machine learning models, especially large language models, to have capabilities by proposing a philosophical and operational framework called CAMA, aiding evaluation and comparison.

Contribution

It introduces the conditional analysis of model abilities (CAMA), a novel framework for understanding and evaluating ML model capabilities in a rigorous, operational manner.

Findings

CAMA provides a clear criterion for model capabilities based on success likelihood.

The framework helps interpret evaluation practices of large language models.

CAMA suggests procedures for fair comparison of different models.

Abstract

What can contemporary machine learning (ML) models do? Given the proliferation of ML models in society, answering this question matters to a variety of stakeholders, both public and private. The evaluation of models' capabilities is rapidly emerging as a key subfield of modern ML, buoyed by regulatory attention and government grants. Despite this, the notion of an ML model possessing a capability has not been interrogated: what are we saying when we say that a model is able to do something? And what sorts of evidence bear upon this question? In this paper, we aim to answer these questions, using the capabilities of large language models (LLMs) as a running example. Drawing on the large philosophical literature on abilities, we develop an account of ML models' capabilities which can be usefully applied to the nascent science of model evaluation. Our core proposal is a conditional analysis of model abilities (CAMA): crudely, a machine learning model has a capability to X just when it would reliably succeed at doing X if it 'tried'. The main contribution of the paper is making this proposal precise in the context of ML, resulting in an operationalisation of CAMA applicable to LLMs. We then put CAMA to work, showing that it can help make sense of various features of ML model evaluation practice, as well as suggest procedures for performing fair inter-model comparisons.