iCLP: Large Language Model Reasoning with Implicit Cognition Latent Planning

TL;DR

This paper introduces iCLP, a framework enabling large language models to perform implicit reasoning by generating and utilizing latent plans, which improves accuracy, efficiency, and cross-domain generalization in reasoning tasks.

Contribution

iCLP is the first approach to incorporate latent plan representations into LLM reasoning, inspired by human implicit cognition, enhancing reasoning performance and interpretability.

Findings

Significant accuracy improvements on mathematical reasoning tasks

Enhanced efficiency in reasoning processes

Strong cross-domain generalization capabilities

Abstract

Large language models (LLMs), when guided by explicit textual plans, can perform reliable step-by-step reasoning during problem-solving. However, generating accurate and effective textual plans remains challenging due to LLM hallucinations and the high diversity of task-specific questions. To address this, we draw inspiration from human Implicit Cognition (IC), the subconscious process by which decisions are guided by compact, generalized patterns learned from past experiences without requiring explicit verbalization. We propose iCLP, a novel framework that enables LLMs to adaptively generate latent plans (LPs), which are compact encodings of effective reasoning instructions. iCLP first distills explicit plans from existing step-by-step reasoning trajectories. It then learns discrete representations of these plans via a vector-quantized autoencoder coupled with a codebook. Finally, by fine-tuning LLMs on paired latent plans and corresponding reasoning steps, the models learn to perform implicit planning during reasoning. Experimental results on mathematical reasoning and code generation tasks demonstrate that, with iCLP, LLMs can plan in latent space while reasoning in language space. This approach yields significant improvements in both accuracy and efficiency and, crucially, demonstrates strong cross-domain generalization while preserving the interpretability of chain-of-thought reasoning.