RM-PoT: Reformulating Mathematical Problems and Solving via Program of Thoughts

TL;DR

RM-PoT enhances large language models' mathematical reasoning by reformulating problems, retrieving relevant examples, and generating executable code, thereby improving robustness and accuracy in solving complex numerical tasks.

Contribution

The paper introduces RM-PoT, a novel three-stage framework combining problem reformulation, code-aided reasoning, and domain-aware few-shot learning to improve mathematical problem solving.

Findings

Reformulating problems reduces structural bias and improves answer accuracy.

Retrieving semantically aligned examples enhances contextual understanding.

Generating executable code leads to more precise numerical solutions.

Abstract

Recently, substantial advancements have been made in training language models to carry out step-by-step reasoning for solving intricate numerical reasoning tasks. Beyond the methods used to solve these problems, the structure and formulation of the problems themselves also play a crucial role in determining the performance of large language models. We observe that even small changes in the surface form of mathematical problems can have a profound impact on both the answer distribution and solve rate. This highlights the vulnerability of LLMs to surface-level variations, revealing its limited robustness when reasoning through complex problems. In this paper, we propose RM-PoT, a three-stage framework that integrates problem reformulation (RM), code-aided reasoning (PoT), and domain-aware few-shot learning to address these limitations. Our approach first reformulates the input problem into diverse surface forms to reduce structural bias, then retrieves five semantically aligned examples from a pre-constructed domain-specific question bank to provide contextual guidance, and finally generates executable Python code for precise computation.