The 17% Gap: Quantifying Epistemic Decay in AI-Assisted Survey Papers

TL;DR

This study quantifies the 17% rate of unverifiable citations in AI survey papers, revealing persistent epistemic decay due to hallucinated references, which threaten scientific reproducibility and trust.

Contribution

It introduces a hybrid forensic pipeline to systematically identify and categorize citation failures, highlighting the stability of high-entropy citation practices in AI literature.

Findings

17% Phantom citation rate detected in recent AI surveys

Three failure modes identified: hallucinations, hallucinated identifiers with valid titles, parsing failures

Citation decay trend remains flat over time, indicating endemic issues

Abstract

The adoption of Large Language Models (LLMs) in scientific writing promises efficiency but risks introducing informational entropy. While "hallucinated papers" are a known artifact, the systematic degradation of valid citation chains remains unquantified. We conducted a forensic audit of 50 recent survey papers in Artificial Intelligence (N=5,514 citations) published between September 2024 and January 2026. We utilized a hybrid verification pipeline combining DOI resolution, Crossref metadata analysis, Semantic Scholar queries, and fuzzy text matching to distinguish between formatting errors ("Sloppiness") and verifiable non-existence ("Phantoms). We detect a persistent 17.0% Phantom Rate -- citations that cannot be resolved to any digital object despite aggressive forensic recovery. Diagnostic categorization reveals three distinct failure modes: pure hallucinations (5.1%), hallucinated identifiers with valid titles (16.4%), and parsing-induced matching failures (78.5%). Longitudinal analysis reveals a flat trend (+0.07 pp/month), suggesting that high-entropy citation practices have stabilized as an endemic feature of the field. The scientific citation graph in AI survey literature exhibits "link rot" at scale. This suggests a mechanism where AI tools act as "lazy research assistants," retrieving correct titles but hallucinating metadata, thereby severing the digital chain of custody required for reproducible science.