MC-NEST: Enhancing Mathematical Reasoning in Large Language Models leveraging a Monte Carlo Self-Refine Tree

TL;DR

MC-NEST is a novel method that enhances large language models' mathematical reasoning by integrating Monte Carlo Tree Search with self-refinement, leading to state-of-the-art performance on complex math benchmarks.

Contribution

It introduces MC-NEST, combining Monte Carlo Tree Search with LLM-based self-evaluation and refinement, significantly improving reasoning capabilities in mathematical tasks.

Findings

Achieves state-of-the-art pass@1 scores on Olympiad benchmarks

Improves solution quality and consistency across different LLMs

Excels in Algebra, Geometry, and Number Theory reasoning

Abstract

Mathematical reasoning presents significant challenges for large language models (LLMs). To enhance their capabilities, we propose Monte Carlo Self-Refine Tree (MC-NEST), an extension of Monte Carlo Tree Search that integrates LLM-based self-refinement and self-evaluation for improved decision-making in complex reasoning tasks. MC-NEST balances exploration and exploitation using Upper Confidence Bound (UCT) scores combined with diverse selection policies. Through iterative critique and refinement, LLMs learn to reason more strategically. Empirical results demonstrate that MC-NEST with an importance sampling policy substantially improves GPT-4o's performance, achieving state-of-the-art pass@1 scores on Olympiad-level benchmarks. Specifically, MC-NEST attains a pass@1 of 38.6 on AIME and 12.6 on MathOdyssey. The solution quality for MC-NEST using GPT-4o and Phi-3-mini reaches 84.0\% and 82.08\%, respectively, indicating robust consistency across different LLMs. MC-NEST performs strongly across Algebra, Geometry, and Number Theory, benefiting from its ability to handle abstraction, logical deduction, and multi-step reasoning -- core skills in mathematical problem solving.