LLMs Should Not Yet Be Credited with Decision Explanation
Wenshuo Wang

TL;DR
This paper argues that LLMs should not yet be credited with providing genuine decision explanations, emphasizing the need for stricter standards to distinguish explanation from prediction and rationalization.
Contribution
It introduces a bridge standard for decision-explanation credit, clarifies the distinction between prediction, rationalization, and explanation, and advocates for calibrated attribution of explanatory claims to LLMs.
Findings
Most evidence supports LLMs' decision prediction and rationale generation, not explanation.
A proposed bridge standard for decision explanation emphasizes targeted, discriminative, and process-sensitive validation.
Lack of strict standards risks premature attribution of explanatory power to LLMs.
Abstract
This position paper argues that LLMs should not yet be credited with decision explanation. This matters because recent work increasingly treats accurate behavioral prediction, plausible rationales, and outcome-conditioned reasoning traces as evidence that LLMs explain why people decide as they do, risking a premature redefinition of what counts as explanatory progress in human decision modeling. We first distinguish three claims with different evidential burdens: decision prediction, rationale generation, and decision explanation. We then argue that the evidence most commonly offered for LLM-based decision accounts directly supports the first two claims, and sometimes explanatory hypothesis generation, but does not distinguish decision explanation from prediction-supportive rationalization. Next, we propose a bridge standard for decision-explanation credit: stronger claims should…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
