Ape210K: A Large-Scale and Template-Rich Dataset of Math Word Problems

TL;DR

Ape210K is a large, diverse, and template-rich dataset of 210,000 Chinese elementary math problems designed to advance the development of math word problem solving systems, highlighting the need for models that incorporate natural language understanding and commonsense knowledge.

Contribution

The paper introduces Ape210K, a significantly larger and more diverse math word problem dataset than previous datasets, and proposes a new seq2seq model that outperforms existing models on this dataset.

Findings

State-of-the-art models perform poorly on Ape210K compared to Math23K.

The proposed copy-augmented seq2seq model improves performance by 3.2%.

Ape210K serves as a challenging benchmark for future research.

Abstract

Automatic math word problem solving has attracted growing attention in recent years. The evaluation datasets used by previous works have serious limitations in terms of scale and diversity. In this paper, we release a new large-scale and template-rich math word problem dataset named Ape210K. It consists of 210K Chinese elementary school-level math problems, which is 9 times the size of the largest public dataset Math23K. Each problem contains both the gold answer and the equations needed to derive the answer. Ape210K is also of greater diversity with 56K templates, which is 25 times more than Math23K. Our analysis shows that solving Ape210K requires not only natural language understanding but also commonsense knowledge. We expect Ape210K to be a benchmark for math word problem solving systems. Experiments indicate that state-of-the-art models on the Math23K dataset perform poorly on Ape210K. We propose a copy-augmented and feature-enriched sequence to sequence (seq2seq) model, which outperforms existing models by 3.2% on the Math23K dataset and serves as a strong baseline of the Ape210K dataset. The gap is still significant between human and our baseline model, calling for further research efforts. We make Ape210K dataset publicly available at https://github.com/yuantiku/ape210k