Practice Makes a Solver Perfect: Data Augmentation for Math Word Problem Solvers

TL;DR

This paper introduces data augmentation techniques to enhance the generalization and robustness of math word problem solvers, significantly improving their performance and ability to handle diverse problems.

Contribution

The paper proposes novel data augmentation methods that increase dataset size fivefold, leading to improved accuracy and robustness of MWP solvers across benchmarks.

Findings

Augmented datasets improve solver accuracy by over five percentage points.

Solvers trained on augmented data perform better on challenge test sets.

Proposed techniques are validated through ablation studies and human evaluation.

Abstract

Existing Math Word Problem (MWP) solvers have achieved high accuracy on benchmark datasets. However, prior works have shown that such solvers do not generalize well and rely on superficial cues to achieve high performance. In this paper, we first conduct experiments to showcase that this behaviour is mainly associated with the limited size and diversity present in existing MWP datasets. Next, we propose several data augmentation techniques broadly categorized into Substitution and Paraphrasing based methods. By deploying these methods we increase the size of existing datasets by five folds. Extensive experiments on two benchmark datasets across three state-of-the-art MWP solvers show that proposed methods increase the generalization and robustness of existing solvers. On average, proposed methods significantly increase the state-of-the-art results by over five percentage points on benchmark datasets. Further, the solvers trained on the augmented dataset perform comparatively better on the challenge test set. We also show the effectiveness of proposed techniques through ablation studies and verify the quality of augmented samples through human evaluation.