Profile Consistency Identification for Open-domain Dialogue Agents

TL;DR

This paper introduces a large-scale dataset and a BERT-based model to identify profile consistency in open-domain dialogue agents, enhancing their ability to maintain coherent attribute profiles during conversations.

Contribution

It provides the first large-scale annotated dataset and a novel key-value structure enriched BERT model for profile consistency identification in dialogue systems.

Findings

The dataset contains over 110K annotated conversations.

The proposed model outperforms strong baselines.

Profile consistency identification improves dialogue coherence.

Abstract

Maintaining a consistent attribute profile is crucial for dialogue agents to naturally converse with humans. Existing studies on improving attribute consistency mainly explored how to incorporate attribute information in the responses, but few efforts have been made to identify the consistency relations between response and attribute profile. To facilitate the study of profile consistency identification, we create a large-scale human-annotated dataset with over 110K single-turn conversations and their key-value attribute profiles. Explicit relation between response and profile is manually labeled. We also propose a key-value structure information enriched BERT model to identify the profile consistency, and it gained improvements over strong baselines. Further evaluations on downstream tasks demonstrate that the profile consistency identification model is conducive for improving dialogue consistency.