Dialogue Language Model with Large-Scale Persona Data Engineering

TL;DR

This paper introduces PPDS, a large-scale persona dialogue system that leverages extensive pre-training, a novel persona extraction model, and augmentation techniques to improve persona consistency and response quality in open-domain dialogue models.

Contribution

The paper presents a new large-scale persona dialogue dataset, a persona extraction model, and augmentation methods to enhance persona consistency in dialogue systems.

Findings

PPDS outperforms baseline models in response quality.

Enhanced persona consistency demonstrated through evaluations.

Effective dataset augmentation reduces invalid persona bias.

Abstract

Maintaining persona consistency is paramount in the application of open-domain dialogue systems, as exemplified by models like ChatGPT. Despite significant advancements, the limited scale and diversity of current persona dialogue datasets remain challenges to achieving robust persona-consistent dialogue models. In this study, drawing inspiration from the success of large-scale pre-training, we introduce PPDS, an open-domain persona dialogue system that employs extensive generative pre-training on a persona dialogue dataset to enhance persona consistency. Specifically, we present a persona extraction model designed to autonomously and precisely generate vast persona dialogue datasets. Additionally, we unveil a pioneering persona augmentation technique to address the invalid persona bias inherent in the constructed dataset. Both quantitative and human evaluations consistently highlight the superior response quality and persona consistency of our proposed model, underscoring its effectiveness.