Fine-Grained Behavior Simulation with Role-Playing Large Language Model on Social Media

TL;DR

This paper introduces FineRob, a detailed social media user behavior dataset, and proposes OM-CoT fine-tuning to improve large language models' ability to simulate user behavior accurately.

Contribution

The paper presents a new fine-grained social media behavior dataset and a novel OM-CoT fine-tuning method to enhance LLMs' behavior simulation capabilities.

Findings

OM-CoT improves behavior simulation accuracy

Two dominant reasoning patterns identified in LLMs

FineRob dataset enables detailed behavioral analysis

Abstract

Large language models (LLMs) have demonstrated impressive capabilities in role-playing tasks. However, there is limited research on whether LLMs can accurately simulate user behavior in real-world scenarios, such as social media. This requires models to effectively analyze a user's history and simulate their role. In this paper, we introduce \textbf{FineRob}, a novel fine-grained behavior simulation dataset. We collect the complete behavioral history of 1,866 distinct users across three social media platforms. Each behavior is decomposed into three fine-grained elements: object, type, and content, resulting in 78.6k QA records. Based on FineRob, we identify two dominant reasoning patterns in LLMs' behavior simulation processes and propose the \textbf{OM-CoT} fine-tuning method to enhance the capability. Through comprehensive experiments, we conduct an in-depth analysis of key factors of behavior simulation and also demonstrate the effectiveness of OM-CoT approach\footnote{Code and dataset are available at \url{https://github.com/linkseed18612254945/FineRob}}