Participatory provenance as representational auditing for AI-mediated public consultation

TL;DR

This paper introduces participatory provenance, a framework for auditing AI-mediated public consultation summaries to ensure they accurately reflect source inputs, revealing significant exclusion and bias in official summaries.

Contribution

It presents a novel measurement framework based on optimal transport, causal inference, and semantic analysis to evaluate and improve AI summaries of public input.

Findings

Official summaries underperform baseline in coverage.

Significant exclusion of dissenting participants.

Factors like brevity and rhetorical style predict representational fidelity.

Abstract

Artificial intelligence is increasingly deployed to synthesize large-scale public input in policy consultations and participatory processes. Yet no formal framework exists for auditing whether these summaries faithfully represent the source population, an accountability gap that existing approaches to AI explainability, grounding and hallucination detection do not address because they focus on output quality rather than input fidelity. Here, participatory provenance is introduced: a measurement framework grounded in optimal transport theory, causal inference and semantic analysis that tracks how individual public submissions are transformed, filtered or lost through AI-mediated summarization. Applied to Canada's 2025-2026 national AI Strategy consultation ($n = 5{,}253$ respondents across two independent policy topics), the framework reveals that both official government summaries underperform a random-participant baseline ($-9.1\%$ and $-8.0\%$ coverage degradation), with $16.9\%$ and $15.3\%$ of participants effectively excluded. Exclusion concentrates in clusters expressing dissent, scepticism and critique of AI ($33$-$88\%$ exclusion rates). Brevity, semantic isolation and rhetorical register independently predict representational outcome. An accompanying open-source interactive tool, the Co-creation Provenance Lab, enables policymakers to audit and iteratively improve summaries, establishing genuine human-in-the-loop oversight at scale.