Boosting Large Language Models with Socratic Method for Conversational Mathematics Teaching

TL;DR

This paper introduces SocraticLLM, a conversational model designed to teach mathematics through Socratic dialogue, supported by a new dataset and techniques that enhance teaching quality and learner engagement.

Contribution

It presents a novel Socratic teaching-based LLM, a high-quality dataset for mathematical teaching, and a knowledge-enhanced baseline that improves problem-solving and educational interactions.

Findings

SocraticLLM outperforms several strong generative models in mathematical teaching tasks.

The SocraticMATH dataset enables effective training and evaluation of Socratic-style educational conversations.

Knowledge-enhanced techniques improve the reliability and guidance quality of the LLM responses.

Abstract

With the introduction of large language models (LLMs), automatic math reasoning has seen tremendous success. However, current methods primarily focus on providing solutions or using techniques like Chain-of-Thought to enhance problem-solving accuracy. In this paper, we focus on improving the capability of mathematics teaching via a Socratic teaching-based LLM (\texttt{SocraticLLM}), which guides learners toward profound thinking with clarity and self-discovery via conversation. We collect and release a high-quality mathematical teaching dataset, named \texttt{SocraticMATH}, which provides Socratic-style conversations of problems with extra knowledge. Also, we propose a knowledge-enhanced LLM as a strong baseline to generate reliable responses with review, guidance/heuristic, rectification, and summarization. Experimental results show the great advantages of \texttt{SocraticLLM} by comparing it with several strong generative models. The codes and datasets are available on \url{https://github.com/ECNU-ICALK/SocraticMath}.