Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing

TL;DR

This paper introduces a novel low-resource domain adaptation method for task-oriented semantic parsing, leveraging BART representations and meta-learning to outperform existing models with significantly less data.

Contribution

It proposes a new approach combining BART-based representation learning and meta-learning for low-resource domain adaptation in semantic parsing.

Findings

Outperforms baseline models with 10-fold less data

Uses BART for improved representation learning

Employs meta-learning for better generalization

Abstract

Task-oriented semantic parsing is a critical component of virtual assistants, which is responsible for understanding the user's intents (set reminder, play music, etc.). Recent advances in deep learning have enabled several approaches to successfully parse more complex queries (Gupta et al., 2018; Rongali et al.,2020), but these models require a large amount of annotated training data to parse queries on new domains (e.g. reminder, music). In this paper, we focus on adapting task-oriented semantic parsers to low-resource domains, and propose a novel method that outperforms a supervised neural model at a 10-fold data reduction. In particular, we identify two fundamental factors for low-resource domain adaptation: better representation learning and better training techniques. Our representation learning uses BART (Lewis et al., 2019) to initialize our model which outperforms encoder-only pre-trained representations used in previous work. Furthermore, we train with optimization-based meta-learning (Finn et al., 2017) to improve generalization to low-resource domains. This approach significantly outperforms all baseline methods in the experiments on a newly collected multi-domain task-oriented semantic parsing dataset (TOPv2), which we release to the public.