Efficient Post-Training Refinement of Latent Reasoning in Large Language Models

TL;DR

This paper introduces a lightweight post-training method to refine latent reasoning in large language models, improving reasoning accuracy without extra training by using contrastive feedback and residual embedding updates.

Contribution

It proposes a novel post-training framework with two strategies for refining latent reasoning trajectories, addressing limitations of existing latent reasoning methods.

Findings

Achieves a 5% accuracy improvement on MathQA

Effective across five reasoning benchmarks

Enhances reasoning without additional training

Abstract

Reasoning is a key component of language understanding in Large Language Models. While Chain-of-Thought prompting enhances performance via explicit intermediate steps, it suffers from sufficient token overhead and a fixed reasoning trajectory, preventing step-wise refinement. Recent advances in latent reasoning address these limitations by refining internal reasoning processes directly in the model's latent space, without producing explicit outputs. However, a key challenge remains: how to effectively update reasoning embeddings during post-training to guide the model toward more accurate solutions. To overcome this challenge, we propose a lightweight post-training framework that refines latent reasoning trajectories using two novel strategies: 1) Contrastive reasoning feedback, which compares reasoning embeddings against strong and weak baselines to infer effective update directions via embedding enhancement; 2) Residual embedding refinement, which stabilizes updates by progressively integrating current and historical gradients, enabling fast yet controlled convergence. Extensive experiments and case studies are conducted on five reasoning benchmarks to demonstrate the effectiveness of the proposed framework. Notably, a 5\% accuracy gain on MathQA without additional training.