Weakly-supervised Semantic Parsing with Abstract Examples

TL;DR

This paper introduces an abstract representation approach to weakly-supervised semantic parsing, significantly improving accuracy on the CNLVR visual reasoning dataset by reducing search complexity and spurious program issues.

Contribution

It proposes using abstract representations with lexical rules to share information across examples, alleviating training difficulties in weak supervision for semantic parsing.

Findings

Achieved 82.5% accuracy on CNLVR, a 14.7% improvement over previous methods.

Demonstrated that abstract representations reduce search space and spuriousness in weakly-supervised training.

First semantic parser for the challenging CNLVR dataset.

Abstract

Training semantic parsers from weak supervision (denotations) rather than strong supervision (programs) complicates training in two ways. First, a large search space of potential programs needs to be explored at training time to find a correct program. Second, spurious programs that accidentally lead to a correct denotation add noise to training. In this work we propose that in closed worlds with clear semantic types, one can substantially alleviate these problems by utilizing an abstract representation, where tokens in both the language utterance and program are lifted to an abstract form. We show that these abstractions can be defined with a handful of lexical rules and that they result in sharing between different examples that alleviates the difficulties in training. To test our approach, we develop the first semantic parser for CNLVR, a challenging visual reasoning dataset, where the search space is large and overcoming spuriousness is critical, because denotations are either TRUE or FALSE, and thus random programs are likely to lead to a correct denotation. Our method substantially improves performance, and reaches 82.5% accuracy, a 14.7% absolute accuracy improvement compared to the best reported accuracy so far.