Disambiguate First, Parse Later: Generating Interpretations for Ambiguity Resolution in Semantic Parsing

TL;DR

This paper presents a modular method for resolving ambiguity in semantic parsing by generating multiple interpretations before mapping to logical forms, leveraging LLM biases and an infilling model to improve coverage.

Contribution

It introduces a novel approach that combines bias exploitation and infilling models to enhance ambiguity resolution in semantic parsing tasks.

Findings

Improved interpretation coverage across datasets

Effective use of LLM biases for initial disambiguation

Generalizes across different annotation styles and database structures

Abstract

Handling ambiguity and underspecification is an important challenge in natural language interfaces, particularly for tasks like text-to-SQL semantic parsing. We propose a modular approach that resolves ambiguity using natural language interpretations before mapping these to logical forms (e.g., SQL queries). Although LLMs excel at parsing unambiguous utterances, they show strong biases for ambiguous ones, typically predicting only preferred interpretations. We constructively exploit this bias to generate an initial set of preferred disambiguations and then apply a specialized infilling model to identify and generate missing interpretations. To train the infilling model, we introduce an annotation method that uses SQL execution to validate different meanings. Our approach improves interpretation coverage and generalizes across datasets with different annotation styles, database structures, and ambiguity types.