Decoupling Knowledge and Reasoning in LLMs: An Exploration Using Cognitive Dual-System Theory

TL;DR

This paper introduces a framework inspired by dual-system theory to separate and analyze the contributions of knowledge and reasoning in large language models, revealing domain-specific behaviors and the effects of scaling.

Contribution

It proposes a novel cognition attribution framework that decouples knowledge and reasoning in LLMs using dual cognitive modes, providing new insights into model behavior and scaling effects.

Findings

Reasoning adjustment benefits reasoning-intensive domains

Parameter scaling improves knowledge and reasoning, especially knowledge

Knowledge is stored mainly in lower layers, reasoning in higher layers

Abstract

While large language models (LLMs) leverage both knowledge and reasoning during inference, the capacity to distinguish between them plays a pivotal role in model analysis, interpretability, and development. Inspired by dual-system cognitive theory, we propose a cognition attribution framework to decouple the contribution of knowledge and reasoning. In particular, the cognition of LLMs is decomposed into two distinct yet complementary phases: knowledge retrieval (Phase 1) and reasoning adjustment (Phase 2). To separate these phases, LLMs are prompted to generate answers under two different cognitive modes, fast thinking and slow thinking, respectively. The performance under different cognitive modes is analyzed to quantify the contribution of knowledge and reasoning. This architecture is employed to 15 LLMs across 3 datasets. Results reveal: (1) reasoning adjustment is domain-specific, benefiting reasoning-intensive domains (e.g., mathematics, physics, and chemistry) and potentially imparing knowledge-intensive domains. (2) Parameter scaling improves both knowledge and reasoning, with knowledge improvements being more pronounced. Additionally, parameter scaling make LLMs reasoning significantly more prudent, while moderately more intelligent. (3) Knowledge primarily resides in lower network layers, while reasoning operates in higher layers. Our framework not only helps understand LLMs from a "decoupling" perspective, but also provides new insights into existing research, including scaling laws, hierarchical knowledge editing, and limitations of small-model reasoning.