Learning by Fixing: Solving Math Word Problems with Weak Supervision

TL;DR

This paper introduces a weakly-supervised learning framework for math word problems that generates diverse solutions by correcting neural network errors through symbolic reasoning, outperforming reinforcement learning baselines.

Contribution

The paper proposes a novel learning-by-fixing framework with tree regularization and memory buffer to improve weakly-supervised math problem solving and solution diversity.

Findings

Outperforms reinforcement learning baselines in weak supervision

Achieves comparable top-1 accuracy to fully-supervised methods

Significantly improves top-3 and top-5 answer accuracy

Abstract

Previous neural solvers of math word problems (MWPs) are learned with full supervision and fail to generate diverse solutions. In this paper, we address this issue by introducing a \textit{weakly-supervised} paradigm for learning MWPs. Our method only requires the annotations of the final answers and can generate various solutions for a single problem. To boost weakly-supervised learning, we propose a novel \textit{learning-by-fixing} (LBF) framework, which corrects the misperceptions of the neural network via symbolic reasoning. Specifically, for an incorrect solution tree generated by the neural network, the \textit{fixing} mechanism propagates the error from the root node to the leaf nodes and infers the most probable fix that can be executed to get the desired answer. To generate more diverse solutions, \textit{tree regularization} is applied to guide the efficient shrinkage and exploration of the solution space, and a \textit{memory buffer} is designed to track and save the discovered various fixes for each problem. Experimental results on the Math23K dataset show the proposed LBF framework significantly outperforms reinforcement learning baselines in weakly-supervised learning. Furthermore, it achieves comparable top-1 and much better top-3/5 answer accuracies than fully-supervised methods, demonstrating its strength in producing diverse solutions.