DOP: Diagnostic-Oriented Prompting for Large Language Models in Mathematical Correction

TL;DR

This paper introduces DOP, a novel prompting method that significantly improves large language models' ability to correct reasoning errors in mathematical problems, emphasizing educational applications over mere solution accuracy.

Contribution

The paper proposes diagnostic-oriented prompting (DOP), a new approach that enhances LLMs' error correction capabilities specifically for mathematical reasoning tasks.

Findings

DOP outperforms existing methods in MWPC tasks

Improved error correction leads to better educational support

Codes and data are publicly available for replication

Abstract

Math world problems correction(MWPC) is a novel task dedicated to rectifying reasoning errors in the process of solving mathematical problems. In this paper, leveraging the advancements in large language models (LLMs), we address two key objectives:(1) Distinguishing between mathematical reasoning and error correction; (2) Exploring strategies to enhance the error correction capabilities of LLMs in mathematics to solve MWPC task. We noticed that, in real-time education,assisting students in recognizing their mistakes is more crucial than simply providing correct answers. However, current research tends to prioritize obtaining accurate solutions to math problems rather than correcting potentially incorrect ones. Therefore, we modify the research paradigm, demonstrating that improving mathematical reasoning abilities does not equate to mastery in error correction. Meanwhile, we propose a novel method called diagnostic-oriented promping(DOP) aimed at facilitating LLMs to excel in error correction. In experiments, DOP has shown outstanding performance, highlighting its significant impact. We argue that in mathematical education, the demand for outstanding correctors surpasses that for proficient reasoners. Codes and data are available on https://github.com/ChenhaoEcnuCS/Reason-Correct.