Let GPT be a Math Tutor: Teaching Math Word Problem Solvers with Customized Exercise Generation

TL;DR

This paper introduces a method where GPT-3 acts as a math tutor to generate personalized exercises, improving smaller student models' accuracy efficiently by focusing on their weaknesses through iterative assessment and training.

Contribution

The paper presents a novel iterative approach for distilling math problem-solving skills from large language models into smaller, personalized student models using tailored exercise generation.

Findings

Outperforms GPT-3 and PaLM in accuracy on three benchmarks

Uses fewer parameters than large models while maintaining high performance

Provides detailed analysis of methodology components

Abstract

In this paper, we present a novel approach for distilling math word problem solving capabilities from large language models (LLMs) into smaller, more efficient student models. Our approach is designed to consider the student model's weaknesses and foster a tailored learning experience by generating targeted exercises aligned with educational science principles, such as knowledge tracing and personalized learning. Concretely, we let GPT-3 be a math tutor and run two steps iteratively: 1) assessing the student model's current learning status on a GPT-generated exercise book, and 2) improving the student model by training it with tailored exercise samples generated by GPT-3. Experimental results reveal that our approach outperforms LLMs (e.g., GPT-3 and PaLM) in accuracy across three distinct benchmarks while employing significantly fewer parameters. Furthermore, we provide a comprehensive analysis of the various components within our methodology to substantiate their efficacy.