Contextual Knowledge Learning For Dialogue Generation

TL;DR

This paper introduces a novel training approach called Contextual Knowledge Learning (CKL) that uses latent vectors to effectively weight conversational context and external knowledge, significantly improving dialogue response quality.

Contribution

It presents a new method for fine-grained weighting of context and knowledge in dialogue models using latent vectors and weak supervision, enhancing response relevance.

Findings

CKL outperforms six strong baseline models.

Improves robustness with smaller training datasets.

Demonstrates significant quality improvements in human evaluations.

Abstract

Incorporating conversational context and knowledge into dialogue generation models has been essential for improving the quality of the generated responses. The context, comprising utterances from previous dialogue exchanges, is used as a source of content for response generation and as a means of selecting external knowledge. However, to avoid introducing irrelevant content, it is key to enable fine-grained scoring of context and knowledge. In this paper, we present a novel approach to context and knowledge weighting as an integral part of model training. We guide the model training through a Contextual Knowledge Learning (CKL) process which involves Latent Vectors for context and knowledge, respectively. CKL Latent Vectors capture the relationship between context, knowledge, and responses through weak supervision and enable differential weighting of context utterances and knowledge sentences during the training process. Experiments with two standard datasets and human evaluation demonstrate that CKL leads to a significant improvement compared with the performance of six strong baseline models and shows robustness with regard to reduced sizes of training sets.