Open-domain Topic Identification of Out-of-domain Utterances using Wikipedia

TL;DR

This paper presents a method for identifying the topics of out-of-domain utterances in spoken dialogue systems by leveraging Wikipedia, improving topic prediction accuracy without sacrificing domain classification performance.

Contribution

The study introduces a novel approach that uses Wikipedia-based external knowledge to enhance out-of-domain utterance topic identification in multi-domain SDS.

Findings

Out-of-domain topic prediction accuracy improved by up to 30%.

The approach maintains domain prediction performance.

Joint training enhances Wikipedia article prediction accuracy.

Abstract

Users of spoken dialogue systems (SDS) expect high quality interactions across a wide range of diverse topics. However, the implementation of SDS capable of responding to every conceivable user utterance in an informative way is a challenging problem. Multi-domain SDS must necessarily identify and deal with out-of-domain (OOD) utterances to generate appropriate responses as users do not always know in advance what domains the SDS can handle. To address this problem, we extend the current state-of-the-art in multi-domain SDS by estimating the topic of OOD utterances using external knowledge representation from Wikipedia. Experimental results on real human-to-human dialogues showed that our approach does not degrade domain prediction performance when compared to the base model. But more significantly, our joint training achieves more accurate predictions of the nearest Wikipedia article by up to about 30% when compared to the benchmarks.