Measuring What AI Systems Might Do: Towards A Measurement Science in AI

TL;DR

This paper advocates for a scientific measurement approach to AI capabilities and propensities, emphasizing the importance of causal, dispositional properties over traditional performance metrics.

Contribution

It introduces a principled framework based on philosophy of science and measurement theory to better evaluate AI systems' dispositional properties.

Findings

Current evaluation methods conflate performance with dispositions.

A new framework for measuring AI dispositions is proposed.

Dominant evaluation practices often bypass causal measurement steps.

Abstract

Scientists, policy-makers, business leaders, and members of the public care about what modern artificial intelligence systems are disposed to do. Yet terms such as capabilities, propensities, skills, values, and abilities are routinely used interchangeably and conflated with observable performance, with AI evaluation practices rarely specifying what quantity they purport to measure. We argue that capabilities and propensities are dispositional properties - stable features of systems characterised by counterfactual relationships between contextual conditions and behavioural outputs. Measuring a disposition requires (i) hypothesising which contextual properties are causally relevant, (ii) independently operationalising and measuring those properties, and (iii) empirically mapping how variation in those properties affects the probability of the behaviour. Dominant approaches to AI evaluation, from benchmark averages to data-driven latent-variable models such as Item Response Theory, bypass these steps entirely. Building on ideas from philosophy of science, measurement theory, and cognitive science, we develop a principled account of AI capabilities and propensities as dispositions, show why prevailing evaluation practices fail to measure them, and outline what disposition-respecting, scientifically defensible AI evaluation would require.