Can I Be of Further Assistance? Using Unstructured Knowledge Access to Improve Task-oriented Conversational Modeling

TL;DR

This paper enhances task-oriented dialogue systems by integrating unstructured external knowledge to handle user requests beyond predefined APIs, improving response quality and coverage.

Contribution

It introduces a pipeline with novel data augmentation techniques for knowledge detection and selection, boosting system performance on benchmark datasets.

Findings

Achieved state-of-the-art results on DSTC9 Track 1.

Improved knowledge selection accuracy.

Enhanced end-to-end dialogue response quality.

Abstract

Most prior work on task-oriented dialogue systems are restricted to limited coverage of domain APIs. However, users oftentimes have requests that are out of the scope of these APIs. This work focuses on responding to these beyond-API-coverage user turns by incorporating external, unstructured knowledge sources. Our approach works in a pipelined manner with knowledge-seeking turn detection, knowledge selection, and response generation in sequence. We introduce novel data augmentation methods for the first two steps and demonstrate that the use of information extracted from dialogue context improves the knowledge selection and end-to-end performances. Through experiments, we achieve state-of-the-art performance for both automatic and human evaluation metrics on the DSTC9 Track 1 benchmark dataset, validating the effectiveness of our contributions.