Learning from Multiple Noisy Augmented Data Sets for Better Cross-Lingual Spoken Language Understanding

TL;DR

This paper introduces a denoising training method that leverages multiple models trained on various augmented datasets to improve cross-lingual spoken language understanding in low-resource languages, outperforming previous methods.

Contribution

The paper proposes a novel denoising training approach that uses mutual supervision among models trained on different augmented data to mitigate noise and enhance SLU performance.

Findings

Outperforms state-of-the-art by 3.05 and 4.24 percentage points

Effective noise mitigation in augmented datasets

Improves SLU accuracy in low-resource languages

Abstract

Lack of training data presents a grand challenge to scaling out spoken language understanding (SLU) to low-resource languages. Although various data augmentation approaches have been proposed to synthesize training data in low-resource target languages, the augmented data sets are often noisy, and thus impede the performance of SLU models. In this paper we focus on mitigating noise in augmented data. We develop a denoising training approach. Multiple models are trained with data produced by various augmented methods. Those models provide supervision signals to each other. The experimental results show that our method outperforms the existing state of the art by 3.05 and 4.24 percentage points on two benchmark datasets, respectively. The code will be made open sourced on github.