ComSearch: Equation Searching with Combinatorial Strategy for Solving Math Word Problems with Weak Supervision

TL;DR

This paper introduces ComSearch, a combinatorial search algorithm that efficiently explores the equation space for math word problems under weak supervision, improving pseudo label quality and solving the false-matching problem.

Contribution

The paper presents a novel combinatorial search algorithm that compresses the equation search space and a ranking model to denoise pseudo labels, advancing weakly-supervised math problem solving.

Findings

Achieves state-of-the-art performance in weak supervision setting

Effectively reduces search space by excluding equivalent equations

Addresses false-matching problem with a ranking-based denoising method

Abstract

Previous studies have introduced a weakly-supervised paradigm for solving math word problems requiring only the answer value annotation. While these methods search for correct value equation candidates as pseudo labels, they search among a narrow sub-space of the enormous equation space. To address this problem, we propose a novel search algorithm with combinatorial strategy \textbf{ComSearch}, which can compress the search space by excluding mathematically equivalent equations. The compression allows the searching algorithm to enumerate all possible equations and obtain high-quality data. We investigate the noise in the pseudo labels that hold wrong mathematical logic, which we refer to as the \textit{false-matching} problem, and propose a ranking model to denoise the pseudo labels. Our approach holds a flexible framework to utilize two existing supervised math word problem solvers to train pseudo labels, and both achieve state-of-the-art performance in the weak supervision task.