Representativeness in Statistics, Politics, and Machine Learning

TL;DR

This paper explores the multifaceted concept of representativeness across statistics, politics, and machine learning, analyzing its meanings, normative implications, and role in fairness debates.

Contribution

It provides a historical and normative analysis of representativeness, clarifying its meanings and implications in statistical, political, and machine learning contexts.

Findings

Representativeness has multiple meanings including typicality, match, and coverage.

The concept has a contested history in statistics and politics.

In machine learning, it influences fairness and accountability discussions.

Abstract

Representativeness is a foundational yet slippery concept. Though familiar at first blush, it lacks a single precise meaning. Instead, meanings range from typical or characteristic, to a proportionate match between sample and population, to a more general sense of accuracy, generalizability, coverage, or inclusiveness. Moreover, the concept has long been contested. In statistics, debates about the merits and methods of selecting a representative sample date back to the late 19th century; in politics, debates about the value of likeness as a logic of political representation are older still. Today, as the concept crops up in the study of fairness and accountability in machine learning, we need to carefully consider the term's meanings in order to communicate clearly and account for their normative implications. In this paper, we ask what representativeness means, how it is mobilized socially, and what values and ideals it communicates or confronts. We trace the concept's history in statistics and discuss normative tensions concerning its relationship to likeness, exclusion, authority, and aspiration. We draw on these analyses to think through how representativeness is used in FAccT debates, with emphasis on data, shift, participation, and power.