Low-Resource Compositional Semantic Parsing with Concept Pretraining

TL;DR

This paper introduces a zero-shot and few-shot domain adaptation method for semantic parsing in voice assistants, leveraging concept encoding and pretraining to improve performance with minimal or no new training data.

Contribution

It proposes a novel architecture combining concept encoding and a decoder-focused pretraining approach for low-resource semantic parsing adaptation.

Findings

Outperforms prior methods in few-shot settings on TOPv2 and SNIPS datasets.

Effective zero-shot domain adaptation with minimal domain metadata.

Improved semantic parsing accuracy in low-resource scenarios.

Abstract

Semantic parsing plays a key role in digital voice assistants such as Alexa, Siri, and Google Assistant by mapping natural language to structured meaning representations. When we want to improve the capabilities of a voice assistant by adding a new domain, the underlying semantic parsing model needs to be retrained using thousands of annotated examples from the new domain, which is time-consuming and expensive. In this work, we present an architecture to perform such domain adaptation automatically, with only a small amount of metadata about the new domain and without any new training data (zero-shot) or with very few examples (few-shot). We use a base seq2seq (sequence-to-sequence) architecture and augment it with a concept encoder that encodes intent and slot tags from the new domain. We also introduce a novel decoder-focused approach to pretrain seq2seq models to be concept aware using Wikidata and use it to help our model learn important concepts and perform well in low-resource settings. We report few-shot and zero-shot results for compositional semantic parsing on the TOPv2 dataset and show that our model outperforms prior approaches in few-shot settings for the TOPv2 and SNIPS datasets.