Knowledge Augmented BERT Mutual Network in Multi-turn Spoken Dialogues

TL;DR

This paper introduces a knowledge-augmented BERT mutual network that enhances multi-turn spoken dialogue understanding by leveraging external knowledge and dialogue context, leading to improved intent and slot detection accuracy.

Contribution

It proposes a novel BERT-based joint model with a knowledge attention module and gating mechanism to better model multi-turn dialogue dynamics and incorporate external knowledge.

Findings

Significant improvements over baseline models in two complex dialogue datasets

Effective filtering of irrelevant knowledge triples improves comprehension

Mutual modeling of SLU tasks enhances understanding accuracy

Abstract

Modern spoken language understanding (SLU) systems rely on sophisticated semantic notions revealed in single utterances to detect intents and slots. However, they lack the capability of modeling multi-turn dynamics within a dialogue particularly in long-term slot contexts. Without external knowledge, depending on limited linguistic legitimacy within a word sequence may overlook deep semantic information across dialogue turns. In this paper, we propose to equip a BERT-based joint model with a knowledge attention module to mutually leverage dialogue contexts between two SLU tasks. A gating mechanism is further utilized to filter out irrelevant knowledge triples and to circumvent distracting comprehension. Experimental results in two complicated multi-turn dialogue datasets have demonstrate by mutually modeling two SLU tasks with filtered knowledge and dialogue contexts, our approach has considerable improvements compared with several competitive baselines.