Explicit v.s. Implicit Memory: Exploring Multi-hop Complex Reasoning Over Personalized Information

TL;DR

This paper introduces a new multi-hop reasoning task over personalized information, compares explicit and implicit memory methods, and proposes a hybrid approach to improve multi-hop reasoning in large language models.

Contribution

It defines a novel multi-hop personalized reasoning task, constructs a dataset, evaluates various memory mechanisms, and proposes the HybridMem method to enhance reasoning capabilities.

Findings

HybridMem outperforms individual memory methods.

Explicit and implicit memories have complementary strengths.

The dataset and evaluation framework facilitate future research.

Abstract

In large language model-based agents, memory serves as a critical capability for achieving personalization by storing and utilizing users' information. Although some previous studies have adopted memory to implement user personalization, they typically focus on preference alignment and simple question-answering. However, in the real world, complex tasks often require multi-hop reasoning on a large amount of user information, which poses significant challenges for current memory approaches. To address this limitation, we propose the multi-hop personalized reasoning task to explore how different memory mechanisms perform in multi-hop reasoning over personalized information. We explicitly define this task and construct a dataset along with a unified evaluation framework. Then, we implement various explicit and implicit memory methods and conduct comprehensive experiments. We evaluate their performance on this task from multiple perspectives and analyze their strengths and weaknesses. Besides, we explore hybrid approaches that combine both paradigms and propose the HybridMem method to address their limitations. We demonstrate the effectiveness of our proposed model through extensive experiments. To benefit the research community, we release this project at https://github.com/nuster1128/MPR.