ATLAS: Adaptive Test-Time Latent Steering with External Verifiers for Enhancing LLMs Reasoning

TL;DR

ATLAS introduces a dynamic, verifier-guided latent steering method that adaptively enhances large language models' reasoning at inference time, improving accuracy and efficiency without additional training.

Contribution

It is the first to incorporate learned latent verification into test-time steering, enabling per-step and per-example adaptive control of LLM reasoning.

Findings

Outperforms fixed steering baselines in mathematical reasoning tasks

Achieves higher accuracy with reduced token usage

Demonstrates scalable, effective reasoning enhancement

Abstract

Recent work on activation and latent steering has demonstrated that modifying internal representations can effectively guide large language models (LLMs) toward improved reasoning and efficiency without additional training. However, most existing approaches rely on fixed steering policies and static intervention strengths, which limit their robustness across problem instances and often result in over- or under-steering. We propose Adaptive Test-time Latent Steering, called (ATLAS), a task-specific framework that dynamically controls steering decisions at inference time using an external, lightweight latent verifier. Given intermediate hidden states, the verifier predicts the quality of ongoing reasoning and adaptively selects whether and how strongly to apply steering, enabling per-example and per-step adjustment with minimal overhead. To our knowledge, ATLAS is the first method to integrate learned latent verification into test-time steering for enhancing LLMs reasoning. Experiments on multiple mathematical reasoning benchmarks show that ATLAS consistently outperforms both vanilla decoding and fixed steering baselines, achieving higher accuracy while substantially reducing test-time token usage. These results demonstrate that verifier-guided latent adaptation provides an effective and scalable mechanism for controlling reasoning efficiency without sacrificing solution quality. All source code will be publicly available.