Contextualize Knowledge Bases with Transformer for End-to-end Task-Oriented Dialogue Systems
Yanjie Gou, Yinjie Lei, Lingqiao Liu, Yong Dai, Chunxu Shen

TL;DR
This paper introduces COMET, a transformer-based framework that fully contextualizes knowledge base entities within dialogue systems by dynamically focusing on relevant entities and dialogue history, leading to improved performance.
Contribution
The paper proposes a novel Memory Mask mechanism in a transformer framework to better represent KB entities by considering full context, enhancing dialogue system effectiveness.
Findings
COMET outperforms existing methods on benchmark datasets.
Dynamic entity focusing improves response accuracy.
Full contextualization enhances KB reasoning capabilities.
Abstract
Incorporating knowledge bases (KB) into end-to-end task-oriented dialogue systems is challenging, since it requires to properly represent the entity of KB, which is associated with its KB context and dialogue context. The existing works represent the entity with only perceiving a part of its KB context, which can lead to the less effective representation due to the information loss, and adversely favor KB reasoning and response generation. To tackle this issue, we explore to fully contextualize the entity representation by dynamically perceiving all the relevant entities} and dialogue history. To achieve this, we propose a COntext-aware Memory Enhanced Transformer framework (COMET), which treats the KB as a sequence and leverages a novel Memory Mask to enforce the entity to only focus on its relevant entities and dialogue history, while avoiding the distraction from the irrelevant…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSpeech and dialogue systems · Topic Modeling · Natural Language Processing Techniques
MethodsLinear Layer · Absolute Position Encodings · Position-Wise Feed-Forward Layer · Gated Recurrent Unit · Adam · Byte Pair Encoding · Softmax · Multi-Head Attention · Layer Normalization · Dynamic Memory Network
