Automatic Evaluation of Neural Personality-based Chatbots

TL;DR

This paper introduces a new method for evaluating how well neural chatbots can generate responses that reflect different personality traits, aiming to improve naturalness and engagement.

Contribution

It proposes a novel evaluation approach specifically designed to assess personality expression in neural dialogue models.

Findings

The method effectively measures personality trait consistency in generated responses.

It provides insights into the relationship between model responses and targeted personality traits.

The approach can guide the development of more engaging and personalized conversational agents.

Abstract

Stylistic variation is critical to render the utterances generated by conversational agents natural and engaging. In this paper, we focus on sequence-to-sequence models for open-domain dialogue response generation and propose a new method to evaluate the extent to which such models are able to generate responses that reflect different personality traits.