Shared Imagination: LLMs Hallucinate Alike
Yilun Zhou, Caiming Xiong, Silvio Savarese, Chien-Sheng Wu

TL;DR
This paper introduces a novel setting called imaginary question answering (IQA) to explore the shared imagination space of large language models, revealing their surprising ability to answer fictitious questions across models, indicating a common imaginative capability.
Contribution
The paper proposes IQA as a new method to analyze model similarity and demonstrates that different LLMs share a common imaginative space during hallucinations.
Findings
Models can answer each other's fictitious questions successfully.
LLMs operate within a shared imagination space during hallucinations.
Implications for understanding model homogeneity and creativity.
Abstract
Despite the recent proliferation of large language models (LLMs), their training recipes -- model architecture, pre-training data and optimization algorithm -- are often very similar. This naturally raises the question of the similarity among the resulting models. In this paper, we propose a novel setting, imaginary question answering (IQA), to better understand model similarity. In IQA, we ask one model to generate purely imaginary questions (e.g., on completely made-up concepts in physics) and prompt another model to answer. Surprisingly, despite the total fictionality of these questions, all models can answer each other's questions with remarkable success, suggesting a "shared imagination space" in which these models operate during such hallucinations. We conduct a series of investigations into this phenomenon and discuss implications on model homogeneity, hallucination, and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
