RNN based Incremental Online Spoken Language Understanding

TL;DR

This paper introduces RNN-based incremental processing for spoken language understanding, reducing latency in intent detection without sacrificing accuracy by analyzing different RNN architectures and proposing an EOS detector.

Contribution

It presents a novel RNN-based incremental SLU system with a new EOS detector, enabling real-time intent detection with lower latency compared to traditional offline systems.

Findings

Lower latency achieved without accuracy loss

Effective segmentation of transcript streams into sentences

Potential for early intent detection before EOS

Abstract

Spoken Language Understanding (SLU) typically comprises of an automatic speech recognition (ASR) followed by a natural language understanding (NLU) module. The two modules process signals in a blocking sequential fashion, i.e., the NLU often has to wait for the ASR to finish processing on an utterance basis, potentially leading to high latencies that render the spoken interaction less natural. In this paper, we propose recurrent neural network (RNN) based incremental processing towards the SLU task of intent detection. The proposed methodology offers lower latencies than a typical SLU system, without any significant reduction in system accuracy. We introduce and analyze different recurrent neural network architectures for incremental and online processing of the ASR transcripts and compare it to the existing offline systems. A lexical End-of-Sentence (EOS) detector is proposed for segmenting the stream of transcript into sentences for intent classification. Intent detection experiments are conducted on benchmark ATIS, Snips and Facebook's multilingual task oriented dialog datasets modified to emulate a continuous incremental stream of words with no utterance demarcation. We also analyze the prospects of early intent detection, before EOS, with our proposed system.