Reasoning Planning for Language Models

TL;DR

This paper introduces EPIC, a novel framework that improves reasoning method selection in language models by leveraging theoretical accuracy bounds and contrastive learning, leading to better accuracy and efficiency.

Contribution

The paper proposes EPIC, a new ensemble planning framework that learns shared representations to optimize reasoning method selection based on theoretical bounds.

Findings

EPIC outperforms baseline methods in mathematical reasoning tasks.

EPIC reduces computational costs while maintaining high accuracy.

Theoretical bounds effectively guide reasoning method selection.

Abstract

Selecting an appropriate reasoning method for a given query remains a key challenge in language model generation. Existing approaches typically generate multiple candidate responses and use an aggregation strategy to select the output answer, often assuming that more candidate answers yield higher accuracy. We revisit this assumption through a rigorous theoretical analysis, deriving accuracy bounds for standard aggregation methods under fixed generation distributions and candidate sizes. Building on these insights, we introduce EPIC, an Ensemble Planning with Contrastive learning framework to learn a shared representation space that captures both model reasoning abilities and query-method compatibility. EPIC incorporates our probability bounds as a regularizer in a utility-driven optimization that balances accuracy and computational cost. Experiments on diverse mathematical reasoning tasks show that EPIC consistently selects optimal reasoning methods, improving accuracy while reducing computational overhead. Our code can be found at https://github.com/nguyenngocbaocmt02/EPIC.