Crafting Customisable Characters with LLMs: A Persona-Driven Role-Playing Agent Framework

TL;DR

This paper introduces a framework for creating customizable role-playing agents using LLMs, enabling diverse character simulation with high consistency and knowledge accuracy, supported by a new dataset and evaluation benchmarks.

Contribution

The paper presents the Customisable Conversation Agent Framework and the SimsConv dataset, advancing personalized character simulation with improved consistency and realism in LLM-based agents.

Findings

SimsChat outperforms existing models in character consistency

Demonstrates high knowledge accuracy in role-playing dialogues

Enables flexible customization of characters and interactions

Abstract

Large Language Models (LLMs) demonstrate remarkable ability to comprehend instructions and generate human-like text, enabling sophisticated agent simulation beyond basic behavior replication. However, the potential for creating freely customisable characters remains underexplored. We introduce the Customisable Conversation Agent Framework, which employs LLMs to simulate real-world characters through personalised characteristic feature injection, enabling diverse character creation according to user preferences. We propose the SimsConv dataset, comprising 68 customised characters and 13,971 multi-turn role-playing dialogues across 1,360 real-world scenes. Characters are initially customised using pre-defined elements (career, aspiration, traits, skills), then expanded through personal and social profiles. Building on this, we present SimsChat, a freely customisable role-playing agent incorporating various realistic settings and topic-specified character interactions. Experimental results on both SimsConv and WikiRoleEval datasets demonstrate SimsChat's superior performance in maintaining character consistency, knowledge accuracy, and appropriate question rejection compared to existing models. Our framework provides valuable insights for developing more accurate and customisable human simulacra. Our data and code are publicly available at https://github.com/Bernard-Yang/SimsChat.