Solving Arithmetic Word Problems by Scoring Equations with Recursive Neural Networks

TL;DR

This paper introduces a novel Tree-RNN approach for scoring candidate equations in arithmetic word problems, leveraging their structure to improve accuracy over previous methods, especially on complex reasoning tasks.

Contribution

The paper proposes a Tree-RNN based method for scoring equations that captures their structure, outperforming sequential models and previous state-of-the-art in accuracy.

Findings

Improves accuracy by over 3% compared to previous methods.

Achieves over 15% improvement on complex reasoning problems.

Outperforms sequential LSTMs by 4% on complex problems.

Abstract

Solving arithmetic word problems is a cornerstone task in assessing language understanding and reasoning capabilities in NLP systems. Recent works use automatic extraction and ranking of candidate solution equations providing the answer to arithmetic word problems. In this work, we explore novel approaches to score such candidate solution equations using tree-structured recursive neural network (Tree-RNN) configurations. The advantage of this Tree-RNN approach over using more established sequential representations, is that it can naturally capture the structure of the equations. Our proposed method consists of transforming the mathematical expression of the equation into an expression tree. Further, we encode this tree into a Tree-RNN by using different Tree-LSTM architectures. Experimental results show that our proposed method (i) improves overall performance with more than 3% accuracy points compared to previous state-of-the-art, and with over 15% points on a subset of problems that require more complex reasoning, and (ii) outperforms sequential LSTMs by 4% accuracy points on such more complex problems.