Gap-Filling Prompting Enhances Code-Assisted Mathematical Reasoning

TL;DR

This paper introduces Gap-Filling Prompting, a two-step method that improves small language models' ability to solve math problems by identifying and filling informational gaps, leading to better reasoning performance.

Contribution

The paper proposes a novel Gap-Filling Prompting strategy that enhances small language models' mathematical reasoning by explicitly addressing informational gaps in problem statements.

Findings

GFP significantly improves SLMs' reasoning accuracy.

GFP outperforms existing prompting methods on benchmark datasets.

Filling informational gaps leads to clearer problem understanding.

Abstract

Despite the strong performance of large language models (LLMs) in tasks like mathematical reasoning, their practical use is limited by high computational demands and proprietary restrictions. Chain-of-thought (CoT) and program-of-thought (PoT) fine-tuning are common methods to transfer LLM knowledge to small language models (SLMs). However, CoT often leads to calculation errors in SLMs, while PoT has shown more promise. While most PoT-based approaches focus on direct problem-to-code conversion or extracting only the key information from questions and then providing code solution for it, this work emphasizes filling the gaps in the question to clearly illustrate the solution path, which can be challenging for an SLM to understand when such information is not explicitly provided. Therefore, this paper introduces Gap-Filling Prompting (GFP), a novel two-step prompting strategy designed to enhance the problem-solving process for SLMs. The first step identifies these gaps and provides hints for filling them, while the second step adds the hints to the question to generate a final code solution. Experimental results on two benchmark datasets demonstrate that GFP significantly improves the mathematical reasoning abilities of SLMs.