DAMP: Doubly Aligned Multilingual Parser for Task-Oriented Dialogue

TL;DR

This paper introduces DAMP, a multilingual semantic parser that significantly enhances zero-shot transfer performance for task-oriented dialogue in code-switched languages through dual-stage alignment techniques.

Contribution

The paper proposes a novel doubly aligned multilingual parser using contrastive pretraining and adversarial finetuning, achieving substantial improvements in multilingual semantic parsing.

Findings

DAMP improves transfer performance by up to 81x on multilingual benchmarks.

Contrastive alignment pretraining enhances English and transfer performance.

DAMP outperforms XLM-R and mT5-Large with fewer parameters.

Abstract

Modern virtual assistants use internal semantic parsing engines to convert user utterances to actionable commands. However, prior work has demonstrated that semantic parsing is a difficult multilingual transfer task with low transfer efficiency compared to other tasks. In global markets such as India and Latin America, this is a critical issue as switching between languages is prevalent for bilingual users. In this work we dramatically improve the zero-shot performance of a multilingual and codeswitched semantic parsing system using two stages of multilingual alignment. First, we show that constrastive alignment pretraining improves both English performance and transfer efficiency. We then introduce a constrained optimization approach for hyperparameter-free adversarial alignment during finetuning. Our Doubly Aligned Multilingual Parser (DAMP) improves mBERT transfer performance by 3x, 6x, and 81x on the Spanglish, Hinglish and Multilingual Task Oriented Parsing benchmarks respectively and outperforms XLM-R and mT5-Large using 3.2x fewer parameters.