El Volumen Louder Por Favor: Code-switching in Task-oriented Semantic Parsing

TL;DR

This paper introduces CSTOP, a dataset for Spanish-English code-switching semantic parsing, and proposes data augmentation methods to improve model performance in low-resource settings, significantly narrowing the accuracy gap.

Contribution

The work provides a new dataset for code-switching semantic parsing and novel data augmentation techniques to enhance model performance with limited data.

Findings

Pre-trained cross-lingual models perform well with limited data.

Data augmentation methods improve zero-shot and few-shot parsing accuracy.

Combining augmentation reduces the accuracy gap by two thirds.

Abstract

Being able to parse code-switched (CS) utterances, such as Spanish+English or Hindi+English, is essential to democratize task-oriented semantic parsing systems for certain locales. In this work, we focus on Spanglish (Spanish+English) and release a dataset, CSTOP, containing 5800 CS utterances alongside their semantic parses. We examine the CS generalizability of various Cross-lingual (XL) models and exhibit the advantage of pre-trained XL language models when data for only one language is present. As such, we focus on improving the pre-trained models for the case when only English corpus alongside either zero or a few CS training instances are available. We propose two data augmentation methods for the zero-shot and the few-shot settings: fine-tune using translate-and-align and augment using a generation model followed by match-and-filter. Combining the few-shot setting with the above improvements decreases the initial 30-point accuracy gap between the zero-shot and the full-data settings by two thirds.