Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning

TL;DR

This survey reviews latent Chain-of-Thought reasoning in Large Language Models, emphasizing its potential to enable more flexible and abstract reasoning by embedding reasoning processes in latent spaces instead of explicit language steps.

Contribution

It provides a systematic taxonomy and comprehensive overview of recent methods, highlighting design principles, applications, and challenges in latent CoT reasoning.

Findings

Latent CoT decouples reasoning from explicit language, enabling richer cognitive representations.

Recent methods are categorized from token-wise to layer-wise strategies.

The survey identifies key challenges and future directions in latent CoT research.

Abstract

Large Language Models (LLMs) have shown impressive performance on complex tasks through Chain-of-Thought (CoT) reasoning. However, conventional CoT relies on explicitly verbalized intermediate steps, which constrains its broader applicability, particularly in abstract reasoning tasks beyond language. To address this, there has been growing research interest in \textit{latent CoT reasoning}, where the reasoning process is embedded within latent spaces. By decoupling reasoning from explicit language generation, latent CoT offers the promise of richer cognitive representations and facilitates more flexible, faster inference. This paper aims to present a comprehensive overview of this emerging paradigm and establish a systematic taxonomy. We analyze recent advances in methods, categorizing them from token-wise horizontal approaches to layer-wise vertical strategies. We then provide in-depth discussions of these methods, highlighting their design principles, applications, and remaining challenges. We hope that our survey provides a structured foundation for advancing this promising direction in LLM reasoning. The relevant papers will be regularly updated at https://github.com/EIT-NLP/Awesome-Latent-CoT.