Taming the Tri-Space Tension: ARC-Guided Hallucination Modeling and Control for Text-to-Image Generation
Jianjiang Yang, Ziyan Huang, Yanshu li, Da Peng, Huaiyuan Yao
TL;DR
This paper introduces a novel framework that models and controls hallucinations in text-to-image diffusion models by analyzing a three-axis cognitive tension space, significantly reducing errors while maintaining image quality.
Contribution
It proposes the Hallucination Tri-Space and ARC, a dynamic vector for real-time alignment monitoring, along with TM-ARC, a latent space controller to mitigate hallucinations.
Findings
Significantly reduces hallucinations in T2I models.
Maintains image quality and diversity during hallucination mitigation.
Provides an interpretable, unified framework for understanding generative failures.
Abstract
Despite remarkable progress in image quality and prompt fidelity, text-to-image (T2I) diffusion models continue to exhibit persistent "hallucinations", where generated content subtly or significantly diverges from the intended prompt semantics. While often regarded as unpredictable artifacts, we argue that these failures reflect deeper, structured misalignments within the generative process. In this work, we propose a cognitively inspired perspective that reinterprets hallucinations as trajectory drift within a latent alignment space. Empirical observations reveal that generation unfolds within a multiaxial cognitive tension field, where the model must continuously negotiate competing demands across three key critical axes: semantic coherence, structural alignment, and knowledge grounding. We then formalize this three-axis space as the Hallucination Tri-Space and introduce the Alignment…
Peer Reviews
Decision·ICLR 2026 Conference Withdrawn Submission
The supposed observation that the latent space of generative image models contain 3 distinct subspaces that lead to current shortcomings in generation is novel and of potentially huge impact to the community. The authors demonstrate applicability on UNet and DiT models, showing the generalizability of the approach.
## Presentation and Obfuscation The paper's central concepts are presented in unnecessarily abstruse language. The work introduces a dense, bespoke vocabulary—"Hallucination Tri-Space" , "cognitive alignment tension" , "anisotropic tension" , and "TensionModulator" —that makes the underlying mechanisms seem far more complex than they likely are. This abstraction is not supported by clear, diagnostic visualizations. Figure 2 provides an abstract t-SNE visualization of deviation clusters but d
- The paper offers a compelling reinterpretation of hallucinations as structured trajectory drifts arising from cognitive alignment imbalances. - The Hallucination Tri-Space abstraction is both elegant and intuitively aligned with human reasoning about image misalignment (semantic, structural, factual). - TM-ARC is designed as a feedback control loop integrated into the sampling process — a notable shift from the usual post-hoc alignment or filtering approaches.
- Although ARC correlates with hallucination types, human validation of tension axes (e.g., via annotator agreement with ARC-predicted dominant axis) is missing. - The current evidence for ARC’s causal role in hallucination mitigation is mostly internal and indirect. - The “cognitive tension” terminology risks being metaphorical rather than mathematical; the physical analogies (stress, tension) might obscure the statistical nature of the latent forces. - Evaluation focuses on prompt alignment me
The paper introduces an interesting approach for examining and understanding hallucinations in T2I diffusion-based generation by analyzing latent space trajectories.
- **Clarity and Writing**: The paper suffers from significant clarity issues. It introduces an overload of new terminology (e.g., "cognitive tension," "generative equilibrium") without providing clear definitions, justifications, or references. It includes Notation issues as well: Many equations use or undefined notation (e.g., $f$ in line 147, $n(z)$ in Eq. 3), making them difficult to follow. More importantly, the presentation style: The experimental and analysis sections read more like a tech
* the initial empirical motivation is well described, although main interesting details are reported in appendix A. One particularly appreciates that the inter-annotator agreement is reported (likely) in terms of Flaiss'kappa. * for the TM-ARC model, a significant evaluation is conducted on two known and relevant datasets, namely DrawBench and Pic-a-pic, and compared to methods that vary in their approaches, namely changing the prompt (Prompt-to-Promp), generative semantic nursing (Attend and E
* the quantitative results (Table 3) are reported on average only, without reporting any confidence interval nor even a standard deviation. Given the usually very small gap of performance with baseline, it raises doubt on the interest of the proposed approach with regards to these last. - in particular, if one compares the score of the proposed approach in the ablation study (Table 1) -- with an unknown backbone, see below -- to the those of ARC with SDXL on DrawBench (Table 3) it exhibits a l
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsData Visualization and Analytics