Ex Ante Evaluation of AI-Induced Idea Diversity Collapse

TL;DR

This paper proposes a framework to evaluate the risk of AI systems causing idea diversity collapse in populations, using model-only data and without human interaction, enabling better assessment of population-level creativity impacts.

Contribution

It introduces a novel ex ante benchmarking protocol for AI-induced crowding, modeling ideas as resources and providing actionable insights to reduce diversity collapse.

Findings

Frontier LLMs fall below the no-crowding parity across tasks.

Estimates of crowding risk stabilize with feasible sample sizes.

Targeted generation protocols can mitigate diversity collapse.

Abstract

Creative AI systems are typically evaluated at the level of individual utility, yet creative outputs are consumed in populations: an idea loses value when many others produce similar ones. This creates an evaluation blind spot, as AI can improve individual outputs while increasing population-level crowding. We introduce a human-relative framework for benchmarking AI-induced human diversity collapse without requiring human-AI interaction data, providing an ex ante protocol to estimate crowding risk from model-only generations and matched unaided human baselines. By modeling ideas as congestible resources, we show that source-level crowding is identifiable from within-distribution comparisons, yielding an excess-crowding coefficient $\Delta$ and a human-relative diversity ratio $\rho$. We show that $\rho\ge1$ is the no-excess-crowding parity condition and connect $\Delta$ to an adoption game with exposure-dependent redundancy costs. Across short stories, marketing slogans, and alternative-uses tasks, three frontier LLMs fall below parity across crowding kernels. Estimates stabilize with feasible model-only sample sizes. Importantly, generation-protocol variants show that crowding can be reduced through targeted design, making diversity collapse an actionable, development-time evaluation target for population-aware creative AI.