Exploring Mathematical Extrapolation of Large Language Models with Synthetic Data

TL;DR

This paper demonstrates that fine-tuning large language models with synthetic data can significantly improve their multi-step mathematical reasoning abilities, including generalization to out-of-domain problems.

Contribution

It introduces a novel arithmetical puzzle problem and shows that synthetic data fine-tuning enhances LLMs' reasoning and generalization capabilities.

Findings

Fine-tuned open-llama-3B achieves 0.44 pass@1 on in-domain data.

Models reach 0.33 and 0.35 pass@1 on extended out-of-domain datasets.

Synthetic data fine-tuning improves multi-step reasoning in LLMs.

Abstract

Large Language Models (LLMs) have shown excellent performance in language understanding, text generation, code synthesis, and many other tasks, while they still struggle in complex multi-step reasoning problems, such as mathematical reasoning. In this paper, through a newly proposed arithmetical puzzle problem, we show that the model can perform well on multi-step reasoning tasks via fine-tuning on high-quality synthetic data. Experimental results with the open-llama-3B model on three different test datasets show that not only the model can reach a zero-shot pass@1 at 0.44 on the in-domain dataset, it also demonstrates certain generalization capabilities on the out-of-domain datasets. Specifically, this paper has designed two out-of-domain datasets in the form of extending the numerical range and the composing components of the arithmetical puzzle problem separately. The fine-tuned models have shown encouraging performance on these two far more difficult tasks with the zero-shot pass@1 at 0.33 and 0.35, respectively.