SmBoP: Semi-autoregressive Bottom-up Semantic Parsing

TL;DR

This paper introduces SmBoP, a semi-autoregressive bottom-up semantic parser that improves decoding efficiency and achieves state-of-the-art accuracy on the Spider benchmark by constructing sub-trees in parallel.

Contribution

The paper proposes a novel bottom-up parsing approach for semantic parsing that is more efficient and effective than traditional top-down autoregressive methods.

Findings

2.2x faster decoding time compared to autoregressive models

Approximately 5x faster training time

State-of-the-art 71.1% accuracy on Spider benchmark

Abstract

The de-facto standard decoding method for semantic parsing in recent years has been to autoregressively decode the abstract syntax tree of the target program using a top-down depth-first traversal. In this work, we propose an alternative approach: a Semi-autoregressive Bottom-up Parser (SmBoP) that constructs at decoding step $t$ the top-$K$ sub-trees of height $\leq t$. Our parser enjoys several benefits compared to top-down autoregressive parsing. From an efficiency perspective, bottom-up parsing allows to decode all sub-trees of a certain height in parallel, leading to logarithmic runtime complexity rather than linear. From a modeling perspective, a bottom-up parser learns representations for meaningful semantic sub-programs at each step, rather than for semantically-vacuous partial trees. We apply SmBoP on Spider, a challenging zero-shot semantic parsing benchmark, and show that SmBoP leads to a 2.2x speed-up in decoding time and a $\sim$5x speed-up in training time, compared to a semantic parser that uses autoregressive decoding. SmBoP obtains 71.1 denotation accuracy on Spider, establishing a new state-of-the-art, and 69.5 exact match, comparable to the 69.6 exact match of the autoregressive RAT-SQL+GraPPa.