Unstructured Knowledge Access in Task-oriented Dialog Modeling using Language Inference, Knowledge Retrieval and Knowledge-Integrative Response Generation

TL;DR

This paper presents a pipeline for task-oriented dialog systems that access unstructured knowledge using natural language inference, knowledge retrieval, and generative response models, improving response quality significantly.

Contribution

It introduces three novel subsystems—KDEAK, KnowleDgEFactor, and Ens-GPT—that together enable effective unstructured knowledge access in dialog systems.

Findings

Outperforms baseline with at least 58.77% BLEU-4 improvement.

Demonstrates effective knowledge detection and retrieval in dialog.

Generates high-quality, knowledge-informed responses.

Abstract

Dialog systems enriched with external knowledge can handle user queries that are outside the scope of the supporting databases/APIs. In this paper, we follow the baseline provided in DSTC9 Track 1 and propose three subsystems, KDEAK, KnowleDgEFactor, and Ens-GPT, which form the pipeline for a task-oriented dialog system capable of accessing unstructured knowledge. Specifically, KDEAK performs knowledge-seeking turn detection by formulating the problem as natural language inference using knowledge from dialogs, databases and FAQs. KnowleDgEFactor accomplishes the knowledge selection task by formulating a factorized knowledge/document retrieval problem with three modules performing domain, entity and knowledge level analyses. Ens-GPT generates a response by first processing multiple knowledge snippets, followed by an ensemble algorithm that decides if the response should be solely derived from a GPT2-XL model, or regenerated in combination with the top-ranking knowledge snippet. Experimental results demonstrate that the proposed pipeline system outperforms the baseline and generates high-quality responses, achieving at least 58.77% improvement on BLEU-4 score.