Integrating Arithmetic Learning Improves Mathematical Reasoning in Smaller Models

TL;DR

This paper demonstrates that incorporating a synthetic arithmetic dataset through intermediate fine-tuning or instruction-tuning improves the mathematical reasoning abilities of smaller models, addressing their limitations in arithmetic computations.

Contribution

The study introduces two methods—intermediate fine-tuning and instruction-tuning mixture—to effectively enhance small models' arithmetic reasoning skills using synthetic data.

Findings

Enhanced arithmetic reasoning in small models

Improved performance on multiple reasoning benchmarks

Effective integration of synthetic arithmetic data

Abstract

While large models pre-trained on high-quality data exhibit excellent performance on mathematical reasoning (e.g., GSM8k, MultiArith), it remains challenging to specialize smaller models for these tasks. Common approaches to address this challenge include knowledge distillation from large teacher models and data augmentation (e.g., rephrasing questions and generating synthetic solutions). Despite these efforts, smaller models struggle with arithmetic computations, leading to errors in mathematical reasoning. In this work, we leverage a synthetic arithmetic dataset generated programmatically to enhance the reasoning capabilities of smaller models. We investigate two key approaches to incorporate this dataset: (1) intermediate fine-tuning, in which a model is fine-tuned on the arithmetic dataset before training it on a reasoning dataset, and (2) integrating the arithmetic dataset into an instruction-tuning mixture, allowing the model to learn arithmetic skills alongside general instruction-following abilities. Our experiments on multiple reasoning benchmarks demonstrate that incorporating an arithmetic dataset, whether through targeted fine-tuning or within an instruction-tuning mixture, enhances models' arithmetic capabilities, thereby improving their mathematical reasoning performance.