PUB: An LLM-Enhanced Personality-Driven User Behaviour Simulator for Recommender System Evaluation

TL;DR

This paper introduces PUB, an LLM-based user behaviour simulator that models personalised behaviour using Big Five traits, improving recommender system evaluation by generating realistic, diverse synthetic data aligned with real-world patterns.

Contribution

The paper presents PUB, a novel personality-driven simulation framework that dynamically infers user traits and generates high-fidelity behavioural data for recommender system testing.

Findings

PUB-generated logs closely match real user behaviour

Reveals meaningful links between personality traits and recommendations

Enhances evaluation scalability and realism

Abstract

Traditional offline evaluation methods for recommender systems struggle to capture the complexity of modern platforms due to sparse behavioural signals, noisy data, and limited modelling of user personality traits. While simulation frameworks can generate synthetic data to address these gaps, existing methods fail to replicate behavioural diversity, limiting their effectiveness. To overcome these challenges, we propose the Personality-driven User Behaviour Simulator (PUB), an LLM-based simulation framework that integrates the Big Five personality traits to model personalised user behaviour. PUB dynamically infers user personality from behavioural logs (e.g., ratings, reviews) and item metadata, then generates synthetic interactions that preserve statistical fidelity to real-world data. Experiments on the Amazon review datasets show that logs generated by PUB closely align with real user behaviour and reveal meaningful associations between personality traits and recommendation outcomes. These results highlight the potential of the personality-driven simulator to advance recommender system evaluation, offering scalable, controllable, high-fidelity alternatives to resource-intensive real-world experiments.