EntroCoT: Enhancing Chain-of-Thought via Adaptive Entropy-Guided Segmentation

TL;DR

EntroCoT introduces an entropy-guided segmentation and evaluation framework to improve the quality of reasoning traces in Large Language Models, leading to better mathematical reasoning performance.

Contribution

The paper presents a novel entropy-based segmentation and Monte Carlo evaluation method to refine reasoning traces, reducing hallucinations and improving dataset quality.

Findings

Outperforms baselines on mathematical benchmarks

Constructs higher-quality reasoning datasets

Enhances reasoning accuracy through refined supervision

Abstract

Chain-of-Thought (CoT) prompting has significantly enhanced the mathematical reasoning capabilities of Large Language Models. We find existing fine-tuning datasets frequently suffer from the "answer right but reasoning wrong" probelm, where correct final answers are derived from hallucinated, redundant, or logically invalid intermediate steps. This paper proposes EntroCoT, a unified framework for automatically identifying and refining low-quality CoT supervision traces. EntroCoT first proposes an entropy-based mechanism to segment the reasoning trace into multiple steps at uncertain junctures, and then introduces a Monte Carlo rollout-based mechanism to evaluate the marginal contribution of each step. By accurately filtering deceptive reasoning samples, EntroCoT constructs a high-quality dataset where every intermediate step in each reasoning trace facilitates the final answer. Extensive experiments on mathematical benchmarks demonstrate that fine-tuning on the subset constructed by EntroCoT consistently outperforms the baseslines of full-dataset supervision.