AdaDec: A Uncertainty-Guided Lookahead Decoding Framework for LLM-Based Code Generation

TL;DR

AdaDec is an adaptive decoding framework that uses token uncertainty to improve code generation accuracy in large language models, outperforming traditional methods.

Contribution

It introduces a novel uncertainty-guided lookahead decoding method that learns model-specific thresholds and selectively reranks tokens at uncertain decision points.

Findings

AdaDec improves Pass@1 accuracy by up to 20.9% over greedy decoding.

It outperforms both Beam Search and AdapT in experiments.

AdaDec maintains efficiency by applying reranking only at high-uncertainty steps.

Abstract

Code generation with large language models (LLMs) is highly sensitive to token selection during decoding, particularly at uncertain decision points that influence program logic. While standard strategies such as greedy decoding treat all tokens uniformly, they overlook code-specific uncertainty patterns, leading to suboptimal performance. This paper presents an empirical study revealing that many generation errors stem from token ranking mistakes at high-uncertainty steps, where the correct token is present but not top-ranked. Motivated by these findings, we propose AdaDec, a lookahead-based uncertainty-guided adaptive decoding framework that integrates a token-level pause-then-rerank mechanism driven by token uncertainty. AdaDec learns model-specific uncertainty thresholds and applies a lookahead-based reranking strategy when uncertainty is high. Experiments on HumanEval+, MBPP+, and DevEval benchmarks show that AdaDec improves Pass@1 accuracy by up to 20.9% in absolute terms over greedy decoding. More importantly, it consistently outperforms both competitive baselines like Beam Search and state-of-the-art adaptive decoding methods such as AdapT, while maintaining high efficiency through selective, uncertainty-triggered pausing. Our results highlight the promise of uncertainty-aware adaptive decoding for improving both the reliability and efficiency of LLM-based code generation.