Think Twice Before You Write -- an Entropy-based Decoding Strategy to Enhance LLM Reasoning

TL;DR

This paper introduces an entropy-guided decoding strategy for large language models that adaptively focuses computation on uncertain tokens, improving reasoning accuracy efficiently.

Contribution

It proposes a novel entropy-based decoding framework with token-level adaptivity and a dynamic stopping criterion, enhancing LLM reasoning with reduced computational costs.

Findings

Achieves strong accuracy on GSM8K and AMC2023 datasets.

Performs comparably to GPT-5 on smaller models at lower cost.

Effectively concentrates computation on high-uncertainty tokens.

Abstract

Decoding strategies play a central role in shaping the reasoning ability of large language models (LLMs). Traditional methods such as greedy decoding and beam search often suffer from error propagation, while sampling-based approaches introduce randomness without adequate robustness. Self-consistency improves reliability by aggregating multiple rollouts, but incurs significant computational overhead. We propose an entropy-guided decoding framework that introduces token-level adaptivity into generation. At each step, the model computes the entropy of the token distribution, identifies high-uncertainty positions, and selectively branches on these vulnerable points. A dynamic pool of partial rollouts is maintained and expanded until solutions are completed, concentrating computation where uncertainty is greatest and avoiding unnecessary exploration in confident regions. To enable efficient termination, we apply a rollout-level Entropy After </Think> (EAT) stopping criterion by performing entropy evaluation after the full reasoning trace, rather than incrementally at every step. Experiments on GSM8K, AMC2023, and their perturbed variants demonstrate that our method achieves consistently strong accuracy. Notably, on smaller LLMs, performance is comparable to GPT-5 while operating at a fraction of the cost.