Measuring Reasoning Utility in LLMs via Conditional Entropy Reduction

TL;DR

This paper investigates how the utility of intermediate reasoning steps in large language models can be measured using conditional entropy, revealing that decreasing entropy correlates with correct answers and enabling early stopping of unproductive reasoning.

Contribution

The study introduces a method to quantify reasoning utility in LLMs via conditional entropy, providing insights into when to halt reasoning to improve accuracy.

Findings

Decreasing conditional entropy over reasoning steps correlates with correct answers.

Incorrect reasoning paths tend to be longer than correct ones.

Conditional entropy can predict the usefulness of reasoning steps for final accuracy.

Abstract

Recent advancements in large language models (LLMs) often rely on generating intermediate reasoning steps to enhance accuracy. However, little work has examined how reasoning utility contributes to the final answer's correctness. Due to the stochastic nature of autoregressive generation, generating more context does not guarantee increased confidence in the answer. If we could predict, during generation, whether a reasoning step will be useful, we could stop early or prune ineffective steps, avoiding distractions in the final decision. We present an oracle study on MATH dataset, using Qwen2.5-32B and GPT-4o to generate reasoning chains, and then employing a separate model (Qwen3-8B) to quantify the utility of these chains for final accuracy. Specifically, we measure the model's uncertainty on the answer span Y at each reasoning step using conditional entropy (expected negative log-likelihood over the vocabulary) with context expanding step by step. Our results show a clear pattern: conditional entropy that decreases over steps is strongly associated with correct answers, whereas flat or increasing entropy often results in wrong answers. We also corroborate that incorrect reasoning paths tend to be longer than correct ones, suggesting that longer reasoning does not necessarily yield better outcomes. These findings serve as a foundation to inspire future work on designing efficient reasoning pipelines that detect and avoid unproductive reasoning early.

Tables

Table 1. Table 1: Statistics of the MATH dataset and a comparison between human and LLM (Qwen2.5-32B) solutions. LLM solutions contain substantially more tokens than human solutions. We report LLM accuracy against human solutions as the ground truth, and observe that lower-accuracy categories tend to involve longer reasoning.

	Counting & Probability	Number Theory	Prealgebra	Algebra	Intermediate Algebra	Precalculus	Geometry
No. of problems	469	540	864	1185	903	546	479
Avg. No. of tokens in human solutions	466	472	357	370	660	780	726
Avg. steps (${\overline{K}}_{\text{gt}}$) of human solutions	5	5	5	4	7	7	8
Avg. No. of tokens in LLM solutions	1728	1565	1281	1306	1860	1930	1895
Avg. steps (${\overline{K}}_{\text{llm}}$) of LLM solutions	9	10	9	10	11	10	10
LLM Accuracy	0.81	0.84	0.92	0.95	0.63	0.63	0.67

Equations

$$X=[{\mathbf{e}}_1,\dots ,{\mathbf{e}}_N]\in {\mathbb{R}}^{N\times d}.$$

$$p_t(v)=\mathrm{Pr}(v\mid C_{<t}),v\in \mathcal{V}.$$

$$R=(z_1,z_2,\dots ,z_L,y_1,y_2,\dots ,y_M)$$

$$P(Y,Z|X)=P(Z|X)\cdot P(Y|X,Z).$$

$$I(Y;Z|X)=H(Y\mid X)-H(Y\mid X,Z).$$

$$p_t(v)=\mathrm{Pr}(v\mid C,y_{<t}),v\in \mathcal{V}.$$

$$H_t=-\sum _{v\in \mathcal{V}}{p}_t(v)\log p_t(v).$$

$$H(Y\mid C)=\frac{1}{|Y|}\sum _{t=1}^{|Y|}{H}_t={\mathbb{E}}_{t\in 1,\dots ,|Y|}\left[H_t\right].$$

$$\text{CE}(Y\mid C_k)=\frac{1}{|Y|}\sum _{t=1}^{|Y|}-\log p(y_t\mid C_k,y_{<t}).$$

$$45,90,135,\dots ,990=1\cdot 45,2\cdot 45,3\cdot 45,\dots ,22\cdot 45.$$

$$\lfloor \frac{999}{45}\rfloor =\lfloor 22.2\rfloor =22$$

$$n=22-1+1=22$$

$$\text{Probability}=\frac{\text{Number of two-digit multiples}}{\text{Total number of multiples}}=\frac{2}{22}=\frac{1}{11}$$

Full text

We present an oracle study on MATH dataset, using Qwen2.5-32B and GPT-4o to generate reasoning chains, and then employing a separate model (Qwen3-8B) to quantify the utility of these chains for final accuracy. Specifically, we measure the model’s uncertainty on the answer span $Y$ at each reasoning step using conditional entropy $\mathbb{E}_{v\in\mathcal{V}}[-p(v|C)\log p(v|C)]$ with context $C$ expanding step by step. Our results show a clear pattern: conditional entropy that decreases over steps is strongly associated with correct answers, whereas flat or increasing entropy often results in wrong answers. We also corroborate that incorrect reasoning paths tend to be longer than correct ones, suggesting that longer reasoning does not necessarily yield better outcomes. These findings serve as a foundation to inspire future work on designing efficient reasoning pipelines that detect and avoid unproductive reasoning early.

Measuring Reasoning Utility in LLMs via Conditional Entropy Reduction

Xu Guo

KTH Royal Institute of Technology, Sweden

xu008@e.ntu.edu.sg

1 Introduction

Recent progress in large language models (LLMs) has been driven largely by scaling model size, yielding unprecedented performance on benchmarks Brown et al. (2020); OpenAI et al. (2024b); Yang et al. (2025). However, scaling benefits show diminishing returns, and further gains from increasing parameters are costly. Meanwhile, such scaling has revealed emergent abilities that can push performance limits without additional training Sun et al. (2024). A notable finding is Chain-of-Thought (CoT) prompting Wei et al. (2022), which prompts the model to always generate intermediate thoughts before providing the answer. The use of CoT has been widely proven effective, inspiring research into advanced reasoning approaches Yao et al. (2023); Xu et al. (2025a, b), agentic frameworks Guo et al. (2024), as well as models that are specially trained to think before answering OpenAI et al. (2024a); DeepSeek-AI et al. (2025).

Yet, unrestricted generation of longer reasoning does not guarantee proportional accuracy gains. Techniques such as self-consistency (Wang et al., 2023) and best-of-N (Lightman et al., 2024) sampling exhibit diminishing returns once the number of sampled chains exceeds a certain threshold (e.g., >40), making them computationally inefficient. If the utility of reasoning for the final answer could be estimated during generation, unproductive reasoning could be halted early, reducing both distractions to the final answer and computational cost.

Prior work on measuring reasoning utility is scarce. Existing approaches fall into two categories: model-based, which rely on a verifier or pretrained Process Reward Model Cobbe et al. (2021); Sun et al. (2024), and metric-based, which use statistical measures such as similarity or confidence. Similarity-based methods primarily target pruning redundancy, which does not directly link to accuracy Zhu et al. (2025); Wang et al. (2025); Xie et al. (2024), as we corroborate in appendix A. Confidence-based metrics, such as perplexity and cross-entropy, are based on $p(y=y_{\mathrm{true}}|C)$, reflecting the model’s confidence in the answer Zhou et al. (2025); Taubenfeld et al. (2025). But our study in appendix B finds limited evidence of its correlation with accuracy. In summary, there is a lack of systematic studies that examined how the model’s uncertainty evolves as reasoning unfolds, and how this trajectory correlates with accuracy.

In this paper, we present an oracle study on the MATH Hendrycks et al. (2021) dataset across seven subjects. We use a reasoning LLM to generate CoT responses, segment them into steps, and compute conditional entropy on the answer span after each step. By grouping the responses into correct and incorrect sets, we plot the entropy trajectories and align them using cubic spline interpolation to reveal general trends across reasoning steps.

Our results on GPT-4o and Qwen2.5-32B reveal a strong correlation: when conditional entropy decreases over steps, the model is more likely to produce the correct answer; when it does not decrease, errors are more common. Also, incorrect reasoning paths are longer on average, indicating that more steps do not necessarily improve outcomes. These findings validate conditional entropy as a promising metric for reasoning utility, providing a foundation for future work on efficient reasoning pipelines that detect and avoid unproductive reasoning early.

2 Preliminary

We consider an autoregressive language model with vocabulary $\mathcal{V}$. Given a natural language prompt $s$, the tokenizer first maps it into a sequence of $N$ token IDs $\mathbf{x}_{1:N}=[x_{1},\dots,x_{N}]$, where $x_{i}\in\mathcal{V}$. Each token ID is mapped to an embedding vector $\mathbf{e}_{i}\in\mathbb{R}^{d}$ through the embedding table $E\in\mathbb{R}^{|\mathcal{V}|\times d}$, producing the embedded input sequence

[TABLE]

In autoregressive generation, the LLM predicts the next token distribution conditioned on a context $C$. Let $C_{t}$ denote the context at step $t$, which initially consists of $X$ and grows as new tokens are generated. For $t$-th token, the model outputs

[TABLE]

where $C_{<t}=[X;R_{<t}]$ and $R_{<t}$ denotes all previously generated tokens. Generation stops when an end-of-sequence token is produced or a predefined maximum length is reached.

Output structure. In our setting, the model’s complete response sequence is denoted as

[TABLE]

where $Z=(z_{1},z_{2},\dots,z_{L})$ is the reasoning chain, consisting of intermediate steps, explanations, or derivations. $Y=(y_{1},y_{2},\dots,y_{M})$ is the final answer span. This explicit decomposition allows us to analyze how the reasoning prefix $Z$ affects the model’s certainty about the final answer $Y$.

3 Measuring the Utility of Reasoning

Given that both the reasoning chain $Z$ and the final answer $Y$ are conditioned on the same input $X$, the joint distribution factorizes as:

[TABLE]

Our key question is: “Given $X$ and a reasoning chain $Z$, how much additional information does $Z$ provide about the correct answer $Y$ beyond what is already contained in $X$?"

Information-theoretic definition: We formalize the utility of $Z$ via conditional mutual information:

[TABLE]

where $H(\cdot)$ denotes entropy. A large reduction in conditional entropy $\Delta H$ indicates that the reasoning chain $Z$ significantly narrows down the correct answer space, while $\Delta H\approx 0$ means $Z$ adds little useful information beyond the question itself.

Practical estimation via conditional entropy: Directly computing $H(Y|X)$ and $H(Y|X,Z)$ is intractable for LLMs. We therefore estimate them using teacher forcing: For each token position $t$ in a target answer sequence $Y=(y_{1},\ldots,y_{|Y|})$ and for a given context $C=X+Z_{\leq k}$ at $k$-th reasoning step, we feed the LLM $[C;y_{<t}]$ to obtain

[TABLE]

The token-level entropy at position $t$ is:

[TABLE]

Since the ground-truth $Y$ is a sequence, we define the sequence-level conditional entropy as the average token entropy:

[TABLE]

This conditional entropy directly reflects the model’s average uncertainty about $Y$ given $C$. Notably, this measurement relies solely on the model’s own output probabilities and requires no additional reward model or external supervision.

Entropy trajectory during reasoning: To study how reasoning affects answer certainty, we decompose $Z$ into $K$ steps and compute $H(Y\mid X,Z_{\leq k})$ for each $k=0,1,\ldots,K$, where $Z_{\leq k}$ denotes the first $k$ reasoning steps. This yields an entropy trajectory that reflects how uncertainty changes as reasoning unfolds. The information gain at $k$-th reasoning step is therefore estimated by $H(Y\mid X,Z_{\leq k-1})-H(Y\mid X,Z_{\leq k})$.

4 Experiment

Dataset.

We use the MATH dataset Hendrycks et al. (2021), which spans seven math domains, across five difficulty levels. Each problem is provided with a human step-by-step solution. Summary statistics are shown in Table 1.

Oracle study setting.

For data generation and inspection, we employ different LLMs. Specifically, we use GPT-4o and Qwen2.5-32B to generate model-produced solutions for each problem and compare them with the human references. To measure uncertainty, we use Qwen3-8B as an inspector that computes answer-span conditional entropy under teacher forcing. We simulate autoregressive generation, and at each token position $t$ in the answer span, we use the last-layer last-token hidden state to obtain the logits for computing the entropy.

Data Processing.

Every reasoning chain can have a different number of steps, making direct averaging on the same problem set infeasible. For each domain, we align entropy trajectories by resampling each chain to a common target steps ($\overline{K}_{\text{gt}}$ for human and $\overline{K}_{\text{llm}}$ for LLM). We use cubic splines for resampling (switch to linear interpolation when steps $\leq 3$) to place all curves on a shared horizontal axis. This up-/down-sampling preserves the overall shape of each trajectory while giving all curves the same number of points. After alignment, we average entropy at each normalised step to obtain domain-level trends, as shown in Figure 1.

5 Results Discussion and Opportunities

This oracle study presents the following insights. Reasoning paths that lead to correct answers consistently achieve information gain over steps. Across seven math categories, the answer-span entropy for correct LLM chains (—) consistently decreases, falls faster, and remains below that of incorrect chains (—). In contrast, incorrect chains show no consistent decrease: on lower-accuracy categories (Fig.1(a)-1(d)) entropy is typically flat or even rises, while on higher-accuracy categories (Fig.1(e)-1(h)) it may decrease yet stays above the correct trajectories. These patterns suggest a simple selection heuristic: prune chains whose entropy does not decrease; if all decrease, rank by the magnitude of the negative slope and keep the top-$k$.

Reasoning paths that lead to wrong answers are typically longer. Across the same problem sets, LLM-generated chains (solid) are generally longer than human chains (dashed), with GPT-4o (Fig.1(a)) exhibiting longer chains than Qwen2.5-32B (Fig.1(b)). Incorrect chains (red) are significantly longer than correct ones (blue). These findings suggest that LLMs often fail on problems that require more steps, and that tackling hard problems requires more advanced reasoning strategies instead of simply scaling the number of steps.

Information gains diminish faster for human reasoning than for LLM reasoning. In human solutions, entropy drops sharply in the early steps and then plateaus; by contrast, correct LLM trajectories (solid blue) decrease entropy more gradually and uniformly across steps. This pattern suggests a strategic difference: humans quickly narrow the answer space and then unravel the intermediate steps, whereas LLMs show weaker early narrowing and progress step by step until the answer becomes clear, as illustrated by an example in Appendix C. This suggests that improving LLMs’ ability to narrow the answer space early, instead of planning step by step, could facilitate efficient reasoning.

Limitations

While our study supports that entropy trajectories provide a model-internal, training-free signal to assess reasoning utility, several limitations remain:

• •

Our analysis relies on access to ground-truth answers, which enables controlled comparison between human and model trajectories. In real deployment, however, ground truth is unavailable, so entropy-based signals must be interpreted without oracle guidance. This limits the direct applicability of our findings, although they still provide useful guidance for pruning parallel scaling strategies (e.g., self-consistency, best-of-$N$) where answer consensus is the main signal.

• •

Experiments are conducted on the MATH dataset. While math problems are a natural testbed for reasoning, we need more dataset evaluations to support whether the observed entropy dynamics generalize to other domains, such as commonsense reasoning.

• •

We use Qwen3-8B as the inspector model. We need more evaluations on different architectures or sizes to confirm the generalizability of the entropy dynamics.

Appendix A Semantic similarity trajectories for Qwen2.5-32B

We study whether cosine similarity between the evolving context and the gold answer span separates correct from incorrect reasoning chains (Figure 2). At each reasoning step $k$, we form $C_{k}=[X;Z_{\leq k}]$ and compute mean-pooled token representations for $C_{k}$ and for the answer span $Y=(y_{1},\dots,y_{M})$. The cosine similarity at step-$k$ is s_{k}=\cos\!\big{(}\bar{\mathbf{e}}(C_{k}),\,\bar{\mathbf{e}}(Y)\big{)}.

We plot $\{s_{k}\}_{k=1}^{K}$ for correct and incorrect chains using the same cubic-spline interpolation to reveal general trends. We find that (1) Similarity trajectories show limited separability between correct and incorrect chains, indicating weak predictive power for accuracy. (2) Nonetheless, $s_{k}$ generally increases with $k$: as reasoning unfolds, the context representation moves closer to the answer-span representation, though the answer is a short numeric expression. This suggests that the semantics of the natural-language derivation for a math problem become progressively closer to the target number in the model’s representation space, although this alignment alone does not determine correctness.

Appendix B Cross-entropy trajectories for Qwen2.5-32B

We also report the cross-entropy trajectory during reasoning (Figure 3). At each reasoning step $k$, we form $C_{k}=[X;Z_{\leq k}]$ and compute:

[TABLE]

The key difference between cross-entropy and entropy is that cross-entropy only aggregates $-\log p(y_{t})$ for the ground-truth tokens, while entropy averages self-information $-p(v)\log p(v)$ across all $v\in\mathcal{V}$. Other commonly used metrics are directly derived from cross-entropy: perplexity is its exponential, and confidence corresponds to $p(y_{t}\mid C_{k},y_{<t})$ for the ground-truth token.

Figure 3 shows that cross-entropy tends to plateau as reasoning unfolds, both for humans and LLMs. Interestingly, we do not observe a consistent difference between correct and incorrect chains. Moreover, LLMs often display lower cross-entropy (i.e., higher confidence) on incorrect chains. This suggests a weak link between confidence and correctness: higher confidence does not reliably imply correctness, raising concerns about confidence-based reasoning evaluation.

Appendix C Examples for human solution and LLM solution

Humans figure out the answer space early; LLMs take it step by step. Here, we illustrate a Number Theory problem reasoning process below, showing the strategic difference in humans and LLMs. This difference is reflected in the entropy trajectories in Figure 1 and we discussed this insight in Section 5.

Number Theory Problem

A positive multiple of 45 less than 1000 is randomly selected. What is the probability that it is a two-digit integer? Express your answer as a common fraction.

Below is the Human Solution:

 

The positive multiples of 45 are

[TABLE]

There are 22 multiples on this list. Every positive multiple of 45 less than 1000 is either a two-digit integer or a three-digit integer. Out of the $99-10+1=90$ two-digit integers, $45$ and $90$ are multiples of 45. Therefore, the probability that the selected multiple of 45 has two digits is $2/22=\boxed{\frac{1}{11}}$.

 

Below is the Solution generated by Qwen2.5-34B:

 

Let’s solve this math problem step by step. Write the answer in .

Problem: A positive multiple of 45 less than 1000 is randomly selected. What is the probability that it is a two-digit integer? Express your answer as a common fraction.

Solution: To determine the probability that a randomly selected positive multiple of 45 less than 1000 is a two-digit integer, we need to follow these steps:

1. Identify the range of multiples of 45 less than 1000:

• •

The smallest multiple of 45 is $45\times 1=45$.

• •

The largest multiple of 45 less than 1000 can be found by dividing 999 by 45 and taking the integer part:

[TABLE]

• •

Therefore, the largest multiple of 45 less than 1000 is $45\times 22=990$.

2. Count the total number of multiples of 45 less than 1000:

• •

These multiples form an arithmetic sequence: $45,90,135,\ldots,990$.

• •

The number of terms in this sequence is given by:

[TABLE]

3. Identify the two-digit multiples of 45:

• •

The smallest two-digit multiple of 45 is $45\times 1=45$.

• •

The largest two-digit multiple of 45 is $45\times 2=90$.

• •

The two-digit multiples of 45 are $45$ and $90$.

4. Count the number of two-digit multiples of 45:

• •

There are 2 two-digit multiples of 45.

5. Calculate the probability:

• •

The probability is the ratio of the number of two-digit multiples to the total number of multiples of 45 less than 1000:

[TABLE]

Thus, the probability that a randomly selected positive multiple of 45 less than 1000 is a two-digit integer is $\boxed{\frac{1}{11}}$.