Multi-chain Graph Refinement and Selection for Reliable Reasoning in Large Language Models

TL;DR

This paper introduces MGRS, a novel framework that generates, refines, and selects the most reliable reasoning paths in large language models, significantly improving accuracy and efficiency across multiple tasks.

Contribution

MGRS is a new reasoning framework that enhances LLM reasoning by generating diverse trajectories, refining responses, and selecting the most reliable answer through graph-based success estimation.

Findings

Achieves an average accuracy of 82.9% across six datasets.

Attains 100% accuracy on the 24-point game.

Provides a 13.6x speed-up over existing methods.

Abstract

The complex reasoning ability of Large Language Models (LLMs) poses a critical bottleneck for their practical applications. Test-time expansion methods such as Tree-of-Thought (ToT) and Graph-of-Thought (GoT) enhance reasoning by introducing intermediate reasoning structures, tree search, or graph-based exploration mechanisms. However, their reasoning strategies suffer from limited diversity, redundant search branches, and inadequate integration and error correction across heterogeneous reasoning paths. To address these limitations, we propose a novel reasoning framework called Multi-chain Graph Refinement & Selection (MGRS), which first generates multiple diverse reasoning trajectories for a given problem, refines candidate responses using a composite self- and cross-verification strategy, then constructs a reasoning relation graph and estimates the success rate of intermediate nodes, and finally computes cumulative success rates to select the most reliable answer and corresponding reasoning trajectory. Experimental results demonstrate that MGRS significantly advances both the reasoning capability and computational efficiency of reasoning enhancement methods. Across six benchmark datasets spanning four distinct tasks, MGRS achieves an average accuracy of 82.9%, outperforming state-of-the-art baselines by a clear margin of 2.1%. Remarkably, on the 24-point game, MGRS attains 100% accuracy for the first time, while delivering a 13.6x speed-up compared to the leading Forest of Thoughts framework.