Multi-task learning to improve natural language understanding

TL;DR

This paper proposes a multi-task learning approach that combines synthetic and real out-of-domain data to enhance neural network-based natural language understanding, demonstrating improved F1-scores in airline travel domain tasks.

Contribution

It introduces a multi-task training method that leverages out-of-domain real data with synthetic data to improve NLU performance.

Findings

F1-score improved from 80.76% to 84.98%.

Multi-task learning enhances generalization to unseen utterances.

Synthetic data alone has limited generalization, improved with real out-of-domain data.

Abstract

Recently advancements in sequence-to-sequence neural network architectures have led to an improved natural language understanding. When building a neural network-based Natural Language Understanding component, one main challenge is to collect enough training data. The generation of a synthetic dataset is an inexpensive and quick way to collect data. Since this data often has less variety than real natural language, neural networks often have problems to generalize to unseen utterances during testing. In this work, we address this challenge by using multi-task learning. We train out-of-domain real data alongside in-domain synthetic data to improve natural language understanding. We evaluate this approach in the domain of airline travel information with two synthetic datasets. As out-of-domain real data, we test two datasets based on the subtitles of movies and series. By using an attention-based encoder-decoder model, we were able to improve the F1-score over strong baselines from 80.76 % to 84.98 % in the smaller synthetic dataset.