Constraint based Knowledge Base Distillation in End-to-End Task Oriented Dialogs

TL;DR

This paper introduces a novel filtering technique for end-to-end task-oriented dialogue systems that improves relevance detection of knowledge base entities, leading to better response generation.

Contribution

It proposes a pairwise similarity filter respecting KB structure and an auxiliary loss, along with a new multiset entity F1 metric, advancing KB entity relevance modeling.

Findings

Outperforms state-of-the-art models on three datasets.

Improves relevance filtering accuracy in KB-based response generation.

Introduces a new metric addressing correctness issues.

Abstract

End-to-End task-oriented dialogue systems generate responses based on dialog history and an accompanying knowledge base (KB). Inferring those KB entities that are most relevant for an utterance is crucial for response generation. Existing state of the art scales to large KBs by softly filtering over irrelevant KB information. In this paper, we propose a novel filtering technique that consists of (1) a pairwise similarity based filter that identifies relevant information by respecting the n-ary structure in a KB record. and, (2) an auxiliary loss that helps in separating contextually unrelated KB information. We also propose a new metric -- multiset entity F1 which fixes a correctness issue in the existing entity F1 metric. Experimental results on three publicly available task-oriented dialog datasets show that our proposed approach outperforms existing state-of-the-art models.