SAND-Math: Using LLMs to Generate Novel, Difficult and Useful Mathematics Questions and Answers

TL;DR

SAND-Math is a pipeline that synthesizes and enhances math problems to improve large language models' reasoning abilities, demonstrating significant performance gains on benchmark datasets.

Contribution

The paper introduces SAND-Math, a novel pipeline for generating and increasing the complexity of math problems to train more capable mathematical LLMs.

Findings

Augmenting with SAND-Math improves benchmark performance by 17.85 points.

Difficulty Hiking increases problem difficulty and model accuracy.

The pipeline provides a scalable toolkit for mathematical reasoning LLMs.

Abstract

The demand for Large Language Models (LLMs) at multiple scales, capable of sophisticated and sound mathematical reasoning, continues to grow. However, the development of performant mathematical LLMs is often bottlenecked by the scarcity of useful training data containing problems with significant complexity. We introduce \textbf{SAND-Math} (\textbf{S}ynthetic \textbf{A}ugmented \textbf{N}ovel and \textbf{D}ifficult Mathematics problems and solutions), a pipeline that addresses this by first synthesizing high-quality problems from scratch and then systematically elevating their complexity via a our newly proposed \textbf{Difficulty Hiking} step. We demonstrate the effectiveness of our approach through two key findings: \textbf{(1)} Augmenting a strong post-training baseline with a small 500-sample SAND-Math dataset significantly boosts performance, outperforming the next-best synthetic dataset by $\uparrow$ 17.85 absolute points on AIME25 benchmark. \textbf{(2)} In a dedicated ablation study, we show the effectiveness of our Difficulty Hiking process in increasing average problem difficulty from 5.02 to 5.98. This step consequently lifts AIME25 results from 46.38\% to 49.23\%. The full generation pipeline, final dataset, and a fine-tuned model form a practical and scalable toolkit for building capable and efficient mathematical reasoning LLMs.