MathDivide: Improved mathematical reasoning by large language models

TL;DR

MathDivide is a novel prompting technique that enhances large language models' mathematical reasoning by decomposing problems into subproblems, using Python code evaluation, and refining answers to improve accuracy.

Contribution

The paper introduces MathDivide, a new prompting method that significantly improves LLM performance on math problems by problem decomposition and code-based evaluation.

Findings

MathDivide outperforms Math-prompter on GSM8K dataset.

It effectively decomposes complex problems into simpler subproblems.

The approach works on both open-source and closed-source LLMs.

Abstract

Large language models have been proven to be capable of handling complex linguistic and cognitive tasks. Therefore their usage has been extended to tasks requiring logical reasoning ability such as Mathematics. In this paper, we propose a prompting technique called MathDivide that breaks down the mathematical problem into simpler subproblems. Each of the subproblems is formulated as an algebraic expression whose value is evaluated by the Python code generated by the LLM for the corresponding algebraic expression. The values fed to the Python code are the numerical values provided in the problem statement. The solutions for the subproblems are composed together to obtain the final answer for the problem statement. Finally, the final answer is compared to the correct answer. If the final answer matches the correct answer, it is produced as output else a refinement prompt is fed to the LLM. We experiment with this prompting technique on both closed-source LLM models and open-source LLM models using GSM8K dataset. The results obtained demonstrate that MathDivide was able to significantly outperform the leading prompting technique called Math-prompter.