Knowledge-Grounded Dialogue with Reward-Driven Knowledge Selection

TL;DR

This paper introduces Knoformer, a reinforcement learning-based model for knowledge-grounded dialogue that automatically selects relevant knowledge without needing labeled data, achieving state-of-the-art results.

Contribution

The paper presents Knoformer, a novel reinforcement learning approach for knowledge selection in dialogue systems that eliminates the need for knowledge labels during training.

Findings

Achieves state-of-the-art performance on two datasets.

Automatically selects relevant knowledge without labeled data.

Outperforms previous models in knowledge-grounded dialogue tasks.

Abstract

Knowledge-grounded dialogue is a task of generating a fluent and informative response based on both conversation context and a collection of external knowledge, in which knowledge selection plays an important role and attracts more and more research interest. However, most existing models either select only one knowledge or use all knowledge for responses generation. The former may lose valuable information in discarded knowledge, while the latter may bring a lot of noise. At the same time, many approaches need to train the knowledge selector with knowledge labels that indicate ground-truth knowledge, but these labels are difficult to obtain and require a large number of manual annotations. Motivated by these issues, we propose Knoformer, a dialogue response generation model based on reinforcement learning, which can automatically select one or more related knowledge from the knowledge pool and does not need knowledge labels during training. Knoformer is evaluated on two knowledge-guided conversation datasets, and achieves state-of-the-art performance.