PersLLM: A Personified Training Approach for Large Language Models

TL;DR

PersLLM introduces a novel framework for enhancing the personification of large language models through improved data construction and dynamic tuning, resulting in more natural and consistent human-like interactions.

Contribution

The paper presents a new approach combining data strategies and automated tuning to better capture and express personality traits in LLMs, addressing previous limitations.

Findings

Enhanced model personality consistency demonstrated by automated metrics.

Improved human evaluation scores for naturalness and opinion expression.

Case studies show potential for diverse human-machine interaction applications.

Abstract

Large language models (LLMs) exhibit human-like intelligence, enabling them to simulate human behavior and support various applications that require both humanized communication and extensive knowledge reserves. Efforts are made to personify LLMs with special training data or hand-crafted prompts, while correspondingly faced with challenges such as insufficient data usage or rigid behavior patterns. Consequently, personified LLMs fail to capture personified knowledge or express persistent opinion. To fully unlock the potential of LLM personification, we propose PersLLM, a framework for better data construction and model tuning. For insufficient data usage, we incorporate strategies such as Chain-of-Thought prompting and anti-induction, improving the quality of data construction and capturing the personality experiences, knowledge, and thoughts more comprehensively. For rigid behavior patterns, we design the tuning process and introduce automated DPO to enhance the specificity and dynamism of the models' personalities, which leads to a more natural opinion communication. Both automated metrics and expert human evaluations demonstrate the effectiveness of our approach. Case studies in human-machine interactions and multi-agent systems further suggest potential application scenarios and future directions for LLM personification.