Entropy-Aware Speculative Decoding Toward Improved LLM Reasoning

TL;DR

This paper introduces Entropy-Aware Speculative Decoding (EASD), a training-free method that improves LLM reasoning by dynamically penalizing low-confidence predictions, enabling surpassing target model performance with efficiency comparable to standard SD.

Contribution

EASD enhances speculative decoding by incorporating entropy-based penalties, allowing the target LLM to outperform the draft model without additional training.

Findings

EASD outperforms existing SD methods on reasoning benchmarks.

EASD often surpasses the performance of the target LLM.

EASD maintains efficiency comparable to standard SD.

Abstract

Speculative decoding (SD) accelerates large language model (LLM) reasoning by using a small draft model to generate candidate tokens, which the target LLM either accepts directly or regenerates upon rejection. However, excessive alignment between the draft and target models constrains SD to the performance of the target LLM. To address this limitation, we propose Entropy-Aware Speculative Decoding (EASD), a training-free enhancement. Building on standard SD, EASD incorporates a dynamic entropy-based penalty. At each decoding step, we employ the entropy of the sampling distribution to quantify model uncertainty. When both models exhibit high entropy with substantial overlap among their top-N predictions, the corresponding token is rejected and re-sampled by the target LLM. This penalty prevents low-confidence errors from propagating. By incorporating draft-model verification, EASD enables the possibility of surpassing the target model's inherent performance. Experiments across multiple reasoning benchmarks demonstrate that EASD consistently outperforms existing SD methods and, in most cases, surpasses the target LLM itself. We further prove that the efficiency of EASD is comparable to that of SD. The code can be found in the Supplementary Materials.