Learning from Executions for Semantic Parsing

TL;DR

This paper introduces a semi-supervised learning approach for semantic parsing that leverages program executability to improve parsing accuracy with limited labeled data.

Contribution

It proposes novel training objectives based on posterior regularization that enhance semi-supervised semantic parsing performance over traditional methods.

Findings

Outperforms conventional beam-search based methods

Bridges the gap between semi-supervised and supervised learning

Effective on datasets like Overnight and GeoQuery

Abstract

Semantic parsing aims at translating natural language (NL) utterances onto machine-interpretable programs, which can be executed against a real-world environment. The expensive annotation of utterance-program pairs has long been acknowledged as a major bottleneck for the deployment of contemporary neural models to real-life applications. In this work, we focus on the task of semi-supervised learning where a limited amount of annotated data is available together with many unlabeled NL utterances. Based on the observation that programs which correspond to NL utterances must be always executable, we propose to encourage a parser to generate executable programs for unlabeled utterances. Due to the large search space of executable programs, conventional methods that use approximations based on beam-search such as self-training and top-k marginal likelihood training, do not perform as well. Instead, we view the problem of learning from executions from the perspective of posterior regularization and propose a set of new training objectives. Experimental results on Overnight and GeoQuery show that our new objectives outperform conventional methods, bridging the gap between semi-supervised and supervised learning.