Mem-PAL: Towards Memory-based Personalized Dialogue Assistants for Long-term User-Agent Interaction

TL;DR

This paper introduces PAL-Bench, a new benchmark and dataset for evaluating and improving long-term personalized dialogue assistants using a hierarchical memory framework, H$^2$Memory, with promising experimental results.

Contribution

It presents PAL-Bench and PAL-Set, the first Chinese dataset for long-term user-agent interactions, and proposes H$^2$Memory, a novel memory framework for personalized dialogue generation.

Findings

H$^2$Memory improves response personalization.

PAL-Bench effectively evaluates long-term interaction capabilities.

The dataset supports future research in personalized dialogue systems.

Abstract

With the rise of smart personal devices, service-oriented human-agent interactions have become increasingly prevalent. This trend highlights the need for personalized dialogue assistants that can understand user-specific traits to accurately interpret requirements and tailor responses to individual preferences. However, existing approaches often overlook the complexities of long-term interactions and fail to capture users' subjective characteristics. To address these gaps, we present PAL-Bench, a new benchmark designed to evaluate the personalization capabilities of service-oriented assistants in long-term user-agent interactions. In the absence of available real-world data, we develop a multi-step LLM-based synthesis pipeline, which is further verified and refined by human annotators. This process yields PAL-Set, the first Chinese dataset comprising multi-session user logs and dialogue histories, which serves as the foundation for PAL-Bench. Furthermore, to improve personalized service-oriented interactions, we propose H$^2$Memory, a hierarchical and heterogeneous memory framework that incorporates retrieval-augmented generation to improve personalized response generation. Comprehensive experiments on both our PAL-Bench and an external dataset demonstrate the effectiveness of the proposed memory framework.