The Global Representativeness Index: A Total Variation Distance Framework for Measuring Demographic Fidelity in Survey Research

TL;DR

The paper introduces the Global Representativeness Index (GRI), a Total Variation Distance-based metric that quantifies how well survey samples match population demographics, addressing a gap in measuring demographic fidelity.

Contribution

It presents a novel, standardized framework and open-source tool for assessing demographic fidelity in survey samples using Total Variation Distance, validated on multiple large-scale surveys.

Findings

Survey samples often have low demographic representativeness scores.

Large probability surveys still show significant demographic gaps.

GRI correlates with classical survey statistics like design effect.

Abstract

Global survey research increasingly informs high-stakes decisions in AI governance and cross-cultural policy, yet no standardized metric quantifies how well a sample's demographic composition matches its target population. Response rates and demographic quotas -- the prevailing proxies for sample quality -- measure effort and coverage but not distributional fidelity. This paper introduces the Global Representativeness Index (GRI), a framework grounded in Total Variation Distance that scores any survey sample against population benchmarks across multiple demographic dimensions on a [0, 1] scale. Validation on seven waves of the Global Dialogues survey (N = 7,500 across 60+ countries) finds fine-grained demographic GRI scores of only 0.33--0.36 -- roughly 43% of the theoretical maximum at that sample size. Cross-validation on the World Values Survey (seven waves, N = 403,000), Afrobarometer Round 9 (N = 53,000), and Latinobarometro (N = 19,000) reveals that even large probability surveys score below 0.22 on fine-grained global demographics when country coverage is limited. The GRI connects to classical survey statistics through the design effect; both metrics are recommended as a minimum summary of sample quality, since GRI quantifies demographic distance symmetrically while effective N captures the asymmetric inferential cost of underrepresentation. The framework is released as an open-source Python library with UN and Pew Research Center population benchmarks, applicable to survey research, machine learning dataset auditing, and AI evaluation benchmarks.