LifeSim: Long-Horizon User Life Simulator for Personalized Assistant Evaluation

TL;DR

LifeSim is a user simulation framework that models long-term user behaviors and cognition to evaluate personalized AI assistants more realistically across diverse scenarios.

Contribution

We introduce LifeSim, a novel user simulator based on the BDI model for realistic long-horizon interactions, and LifeSim-Eval, a comprehensive benchmark for personalized assistant evaluation.

Findings

Current LLMs struggle with implicit intentions.

Long-term user preference modeling remains challenging.

LifeSim reveals limitations of existing models in complex scenarios.

Abstract

The rapid advancement of large language models (LLMs) has accelerated progress toward universal AI assistants. However, existing benchmarks for personalized assistants remain misaligned with real-world user-assistant interactions, failing to capture the complexity of external contexts and users' cognitive states. To bridge this gap, we propose LifeSim, a user simulator that models user cognition through the Belief-Desire-Intention (BDI) model within physical environments for coherent life trajectories generation, and simulates intention-driven user interactive behaviors. Based on LifeSim, we introduce LifeSim-Eval, a comprehensive benchmark for multi-scenario, long-horizon personalized assistance. LifeSim-Eval covers 8 life domains and 1,200 diverse scenarios, and adopts a multi-turn interactive method to assess models' abilities to complete explicit and implicit intentions, recover user profiles, and produce high-quality responses. Under both single-scenario and long-horizon settings, our experiments reveal that current LLMs face significant limitations in handling implicit intention and long-term user preference modeling.