Program of Equations Thoughts to Solve Algebra Word Problems

TL;DR

This paper introduces POET, a two-stage method transforming algebra word problem solving into equation prediction and code generation, significantly improving accuracy by offloading calculations to a Python interpreter, and achieves state-of-the-art results.

Contribution

The paper proposes POET, a novel approach that reduces calculation errors in LLMs by offloading computations to Python, and introduces Zero-shot POET for direct code generation, setting new SOTA results.

Findings

Achieves 95.3% accuracy on PEN dataset

Achieves 98.0% accuracy on ALG514 dataset

Zero-shot POET attains 95.5% on DRAW-1K dataset

Abstract

Solving algebraic word problems (AWPs) has recently emerged as an important natural language processing task. Recently, large language models (LLMs) have demonstrated powerful mathematical capabilities, and the Chain-of-Thought technique, which guides LLMs through step-by-step reasoning, has yielded impressive results. However, this reasoning ability is limited by the computational weaknesses of LLMs themselves, where calculation errors can accumulate, leading to incorrect final answers. To address this, we propose Program of Equations Thoughts (POET), which transforms the task of generating step-by-step reasoning answers into a two-stage task of predicting equations and generating code, offloading complex computations to a Python interpreter to avoid calculation errors in LLMs. Furthermore, we propose Zero-shot POET, which utilizes a manually designed template to enable LLMs to directly generate Python code for one-step solving. Our method achieves accuracies of 95.3% and 98.0% on the PEN and ALG514 datasets, respectively, setting a new state-of-the-art (SOTA). Zero-shot POET also achieves the SOTA result of 95.5% on the DRAW-1K dataset.