Bridging the Know-Act Gap via Task-Level Autoregressive Reasoning

TL;DR

This paper identifies a fundamental gap between recognition and generation in large language models, and introduces DeIllusionLLM, a framework that unifies these modes to improve reasoning and reduce errors in scientific question answering.

Contribution

It presents a novel task-level autoregressive approach with self-distillation to bridge the know-act gap in LLMs, enhancing their ability to handle faulty scientific questions.

Findings

DeIllusionLLM significantly reduces answer-despite-error failures.

The approach maintains strong general reasoning performance.

Self-distillation effectively unifies discriminative and generative reasoning.

Abstract

LLMs often generate seemingly valid answers to flawed or ill-posed inputs. This is not due to missing knowledge: under discriminative prompting, the same models can mostly identify such issues, yet fail to reflect this in standard generative responses. This reveals a fundamental know-act gap between discriminative recognition and generative behavior. Prior work largely characterizes this issue in narrow settings, such as math word problems or question answering, with limited focus on how to integrate these two modes. In this work, we present a comprehensive analysis using FaultyScience, a newly constructed large-scale, cross-disciplinary benchmark of faulty scientific questions. We show that the gap is pervasive and stems from token-level autoregression, which entangles task selection (validate vs. answer) with content generation, preventing discriminative knowledge from being utilized. To address this, we propose DeIllusionLLM, a task-level autoregressive framework that explicitly models this decision. Through self-distillation, the model unifies discriminative judgment and generative reasoning within a single backbone. Empirically, DeIllusionLLM substantially reduces answer-despite-error failures under natural prompting while maintaining general reasoning performance, demonstrating that self-distillation is an effective and scalable solution for bridging the discriminative-generative know-act gap