MDToC: Metacognitive Dynamic Tree of Concepts for Boosting Mathematical Problem-Solving of Large Language Models

TL;DR

MDToC introduces a three-phase metacognitive approach that constructs concept trees and verifies calculations, significantly improving the mathematical reasoning accuracy of large language models across multiple benchmarks.

Contribution

It presents a novel three-phase method, MDToC, that enhances LLMs' mathematical reasoning by constructing concept trees and employing verification, outperforming existing prompting techniques.

Findings

Achieved up to 86.6% accuracy on MATH benchmark.

Outperformed existing prompting methods by up to 7.6%.

Demonstrated effectiveness across multiple LLM backbones.

Abstract

Despite advances in mathematical reasoning capabilities, Large Language Models (LLMs) still struggle with calculation verification when using established prompting techniques. We present MDToC (Metacognitive Dynamic Tree of Concepts), a three-phase approach that constructs a concept tree, develops accuracy-verified calculations for each concept, and employs majority voting to evaluate competing solutions. Evaluations across CHAMP, MATH, and Game-of-24 benchmarks demonstrate our MDToC's effectiveness, with GPT-4-Turbo achieving 58.1\% on CHAMP, 86.6\% on MATH, and 85\% on Game-of-24 - outperforming GoT by 5\%, 5.4\%, and 4\% on all these tasks, respectively, without hand-engineered hints. MDToC consistently surpasses existing prompting methods across all backbone models, yielding improvements of up to 7.6\% over ToT and 6.2\% over GoT, establishing metacognitive calculation verification as a promising direction for enhanced mathematical reasoning.