Using Speech Synthesis to Train End-to-End Spoken Language Understanding Models

TL;DR

This paper introduces a method using speech synthesis to generate synthetic training data, enabling end-to-end spoken language understanding models to be trained without extensive in-domain speech recordings.

Contribution

The paper proposes a novel approach of using speech synthesis for data augmentation in end-to-end SLU models, reducing the need for real speech data.

Findings

Synthetic data improves SLU model performance

Effective as sole training data source

Enhances data diversity and robustness

Abstract

End-to-end models are an attractive new approach to spoken language understanding (SLU) in which the meaning of an utterance is inferred directly from the raw audio without employing the standard pipeline composed of a separately trained speech recognizer and natural language understanding module. The downside of end-to-end SLU is that in-domain speech data must be recorded to train the model. In this paper, we propose a strategy for overcoming this requirement in which speech synthesis is used to generate a large synthetic training dataset from several artificial speakers. Experiments on two open-source SLU datasets confirm the effectiveness of our approach, both as a sole source of training data and as a form of data augmentation.