Meta learning to classify intent and slot labels with noisy few shot examples

TL;DR

This paper introduces a new benchmark for few-shot spoken language understanding that includes noisy data and proposes a noise-robust meta-learning model based on prototypical networks, outperforming existing methods.

Contribution

It defines a novel few-shot robust SLU benchmark with noise types and develops a prototypical network-based model that improves robustness and accuracy.

Findings

The proposed model outperforms fine-tuning and MAML in noisy conditions.

The benchmark includes realistic noise scenarios for SLU tasks.

The model achieves higher intent classification accuracy and slot F1 scores.

Abstract

Recently deep learning has dominated many machine learning areas, including spoken language understanding (SLU). However, deep learning models are notorious for being data-hungry, and the heavily optimized models are usually sensitive to the quality of the training examples provided and the consistency between training and inference conditions. To improve the performance of SLU models on tasks with noisy and low training resources, we propose a new SLU benchmarking task: few-shot robust SLU, where SLU comprises two core problems, intent classification (IC) and slot labeling (SL). We establish the task by defining few-shot splits on three public IC/SL datasets, ATIS, SNIPS, and TOP, and adding two types of natural noises (adaptation example missing/replacing and modality mismatch) to the splits. We further propose a novel noise-robust few-shot SLU model based on prototypical networks. We show the model consistently outperforms the conventional fine-tuning baseline and another popular meta-learning method, Model-Agnostic Meta-Learning (MAML), in terms of achieving better IC accuracy and SL F1, and yielding smaller performance variation when noises are present.