Spoken Language Understanding for Conversational AI: Recent Advances and Future Direction

TL;DR

This paper reviews recent advances in spoken language understanding for conversational AI, focusing on deep learning models like Transformers and BERT, and discusses future directions and practical implementations.

Contribution

It provides a comprehensive overview of joint intent detection and slot filling models, highlighting recent deep learning techniques and offering practical code demonstrations.

Findings

Transformer-based models improve accuracy in intent detection and slot filling.

Pre-trained models like BERT enhance performance on SLU tasks.

Joint modeling boosts overall understanding in conversational AI.

Abstract

When a human communicates with a machine using natural language on the web and online, how can it understand the human's intention and semantic context of their talk? This is an important AI task as it enables the machine to construct a sensible answer or perform a useful action for the human. Meaning is represented at the sentence level, identification of which is known as intent detection, and at the word level, a labelling task called slot filling. This dual-level joint task requires innovative thinking about natural language and deep learning network design, and as a result, many approaches and models have been proposed and applied. This tutorial will discuss how the joint task is set up and introduce Spoken Language Understanding/Natural Language Understanding (SLU/NLU) with Deep Learning techniques. We will cover the datasets, experiments and metrics used in the field. We will describe how the machine uses the latest NLP and Deep Learning techniques to address the joint task, including recurrent and attention-based Transformer networks and pre-trained models (e.g. BERT). We will then look in detail at a network that allows the two levels of the task, intent classification and slot filling, to interact to boost performance explicitly. We will do a code demonstration of a Python notebook for this model and attendees will have an opportunity to watch coding demo tasks on this joint NLU to further their understanding.