Improving Personality Consistency in Conversation by Persona Extending

TL;DR

This paper introduces a retrieval-to-prediction framework for chatbots that enhances personality consistency and addresses the out-of-predefined persona problem by retrieving relevant personas and using a posterior-scored transformer.

Contribution

It proposes a novel retrieval-to-prediction paradigm with a persona retrieval model and a posterior-scored transformer to improve chatbot personality consistency and handle out-of-predefined personas.

Findings

Significant improvements in automatic metrics and human evaluations.

Effective mitigation of the out-of-predefined persona problem.

Introduction of the IT-ConvAI2 dataset highlighting the OOP issue.

Abstract

Endowing chatbots with a consistent personality plays a vital role for agents to deliver human-like interactions. However, existing personalized approaches commonly generate responses in light of static predefined personas depicted with textual description, which may severely restrict the interactivity of human and the chatbot, especially when the agent needs to answer the query excluded in the predefined personas, which is so-called out-of-predefined persona problem (named OOP for simplicity). To alleviate the problem, in this paper we propose a novel retrieval-to-prediction paradigm consisting of two subcomponents, namely, (1) Persona Retrieval Model (PRM), it retrieves a persona from a global collection based on a Natural Language Inference (NLI) model, the inferred persona is consistent with the predefined personas; and (2) Posterior-scored Transformer (PS-Transformer), it adopts a persona posterior distribution that further considers the actual personas used in the ground response, maximally mitigating the gap between training and inferring. Furthermore, we present a dataset called IT-ConvAI2 that first highlights the OOP problem in personalized dialogue. Extensive experiments on both IT-ConvAI2 and ConvAI2 demonstrate that our proposed model yields considerable improvements in both automatic metrics and human evaluations.