SBI-RAG: Enhancing Math Word Problem Solving for Students through Schema-Based Instruction and Retrieval-Augmented Generation

TL;DR

This paper introduces SBI-RAG, a novel framework combining schema-based instruction with retrieval-augmented generation to improve math word problem solving in students, demonstrating enhanced reasoning and structure over existing LLMs.

Contribution

The paper presents SBI-RAG, a new approach integrating schemas with LLMs for better problem-solving in math word problems, emphasizing step-by-step reasoning and educational benefits.

Findings

SBI-RAG improves reasoning clarity in solutions.

It outperforms GPT-3.5 Turbo on GSM8K dataset.

Introduces a new 'reasoning score' metric.

Abstract

Many students struggle with math word problems (MWPs), often finding it difficult to identify key information and select the appropriate mathematical operations. Schema-based instruction (SBI) is an evidence-based strategy that helps students categorize problems based on their structure, improving problem-solving accuracy. Building on this, we propose a Schema-Based Instruction Retrieval-Augmented Generation (SBI-RAG) framework that incorporates a large language model (LLM). Our approach emphasizes step-by-step reasoning by leveraging schemas to guide solution generation. We evaluate its performance on the GSM8K dataset, comparing it with GPT-4 and GPT-3.5 Turbo, and introduce a "reasoning score" metric to assess solution quality. Our findings suggest that SBI-RAG enhances reasoning clarity and facilitates a more structured problem-solving process potentially providing educational benefits for students.