How unique are hallucinated citations offered by generative Artificial Intelligence models?

TL;DR

This study examines how generative AI models produce patterned yet often hallucinated academic citations, revealing risks to scholarly integrity and the limitations of current verification methods.

Contribution

It uncovers the structured nature of hallucinated citations and evaluates AI's tendency to generate plausible but false references without factual verification.

Findings

Nearly 30% of hallucinated citations are duplicates of real references.

ChatGPT reconstructs references from learned patterns rather than factual recall.

9.2% of AI-generated essays contained hallucinated references, including a common phantom citation.

Abstract

This paper investigates how generative AI produces and propagates hallucinated academic references, focusing on the recurring non-existent citation 'Education Governance and Datafication' attributed to Ben Williamson and Nelli Piattoeva. Drawing on 137 accessible source papers identified through Google Scholar and Google searches, the study analyses the structure, recurrence, and onward citation of this phantom reference. It shows that hallucinated citations are not random inventions but patterned recombinations of real authors, journals, dates, and keywords, with duplication occurring in nearly 30% of cases. The paper also reports a structured interrogation of ChatGPT 5-mini about how it generates citations and finds that, absent verification, the model reconstructs plausible references from learned patterns rather than factual recall. Finally, ten AI-generated essays on datafication and school governance were examined: while most references were genuine or partly accurate, 9.2% remained hallucinated, including an exact match to the most common phantom citation. The findings highlight ongoing risks to academic integrity and show that web-enabled AI still does not fully eliminate fabricated references.