Plausibility as Failure: How LLMs and Humans Co-Construct Epistemic Error

TL;DR

This paper explores how human evaluators interpret LLM errors, revealing that superficial plausibility often masks deeper inaccuracies, and emphasizing the relational nature of epistemic failure in human-AI interactions.

Contribution

It introduces a framework for understanding epistemic errors as co-constructed by model plausibility and human interpretive heuristics, shifting focus from predictive metrics to interpretive effects.

Findings

LLM errors shift from predictive to hermeneutic forms

Evaluators rely on surface cues, conflating correctness and relevance

Error detection is hindered by superficial plausibility and cognitive drift

Abstract

Large language models (LLMs) are increasingly used as epistemic partners in everyday reasoning, yet their errors remain predominantly analyzed through predictive metrics rather than through their interpretive effects on human judgment. This study examines how different forms of epistemic failure emerge, are masked, and are tolerated in human AI interaction, where failure is understood as a relational breakdown shaped by model-generated plausibility and human interpretive judgment. We conducted a three round, multi LLM evaluation using interdisciplinary tasks and progressively differentiated assessment frameworks to observe how evaluators interpret model responses across linguistic, epistemic, and credibility dimensions. Our findings show that LLM errors shift from predictive to hermeneutic forms, where linguistic fluency, structural coherence, and superficially plausible citations conceal deeper distortions of meaning. Evaluators frequently conflated criteria such as correctness, relevance, bias, groundedness, and consistency, indicating that human judgment collapses analytical distinctions into intuitive heuristics shaped by form and fluency. Across rounds, we observed a systematic verification burden and cognitive drift. As tasks became denser, evaluators increasingly relied on surface cues, allowing erroneous yet well formed answers to pass as credible. These results suggest that error is not solely a property of model behavior but a co-constructed outcome of generative plausibility and human interpretive shortcuts. Understanding AI epistemic failure therefore requires reframing evaluation as a relational interpretive process, where the boundary between system failure and human miscalibration becomes porous. The study provides implications for LLM assessment, digital literacy, and the design of trustworthy human AI communication.