Searching for Better Database Queries in the Outputs of Semantic Parsers

TL;DR

This paper enhances semantic parsers for database query generation by integrating a search algorithm guided by external criteria, improving the accuracy of generated queries especially for unseen databases.

Contribution

It introduces a search-based method to improve neural semantic parsers using external evaluation criteria, enabling better query correctness and generalization.

Findings

Increased number of correct queries passing test criteria

Improved generalization to unseen databases

Enhanced robustness of semantic parsing models

Abstract

The task of generating a database query from a question in natural language suffers from ambiguity and insufficiently precise description of the goal. The problem is amplified when the system needs to generalize to databases unseen at training. In this paper, we consider the case when, at the test time, the system has access to an external criterion that evaluates the generated queries. The criterion can vary from checking that a query executes without errors to verifying the query on a set of tests. In this setting, we augment neural autoregressive models with a search algorithm that looks for a query satisfying the criterion. We apply our approach to the state-of-the-art semantic parsers and report that it allows us to find many queries passing all the tests on different datasets.