Bridging Internal Probability and Self-Consistency for Effective and Efficient LLM Reasoning

TL;DR

This paper introduces a theoretical analysis of reasoning techniques in large language models, identifies their limitations, and proposes RPC, a method that improves reasoning accuracy and efficiency by combining perplexity and self-consistency.

Contribution

It provides the first theoretical error decomposition of perplexity and self-consistency methods and introduces RPC, a novel approach that enhances reasoning performance and sample efficiency.

Findings

RPC accelerates convergence of estimation error

RPC reduces model error effectively

Empirical results show improved reasoning accuracy

Abstract

Recent advancements in large language models (LLMs) have demonstrated remarkable reasoning capabilities. However, single-shot inference often yields unreliable results for complex reasoning tasks, leading researchers to explore multiple reasoning paths through methods such as perplexity and self-consistency. In this paper, we present the first theoretical error decomposition analysis of these techniques, breaking down their error into estimation error and model error. Our analysis reveals a fundamental trade-off: perplexity methods suffer from substantial model error due to the absence of a proper consistency function, while self-consistency exhibits high estimation error due to a slow error convergence rate. To overcome these limitations, we propose Reasoning-Pruning Perplexity Consistency (RPC). This approach combines Perplexity Consistency, which seamlessly integrates LLM perplexity with self-consistency, and Reasoning Pruning, which eliminates low-probability reasoning paths to effectively prevent the degeneration of estimation error reduction. Theoretical analysis demonstrates that RPC not only accelerates the convergence rate of estimation error to an exponential level but also holds strong potential for further reducing model error. Extensive empirical evaluations on seven benchmark datasets confirm that RPC can significantly improve reasoning performance, sample efficiency, and confidence reliability.