User Profile with Large Language Models: Construction, Updating, and Benchmarking

TL;DR

This paper introduces open-source datasets for user profile construction and updating, and proposes a probabilistic LLM-based method that enhances profile accuracy and context-awareness, validated by experiments with Mistral-7b and Llama2-7b.

Contribution

It provides high-quality datasets for dynamic user profile modeling and demonstrates a novel LLM-based approach for profile construction and updating.

Findings

LLMs improve profile precision and recall.

Datasets enable evaluation of profile modeling techniques.

Proposed method achieves high evaluation scores.

Abstract

User profile modeling plays a key role in personalized systems, as it requires building accurate profiles and updating them with new information. In this paper, we present two high-quality open-source user profile datasets: one for profile construction and another for profile updating. These datasets offer a strong basis for evaluating user profile modeling techniques in dynamic settings. We also show a methodology that uses large language models (LLMs) to tackle both profile construction and updating. Our method uses a probabilistic framework to predict user profiles from input text, allowing for precise and context-aware profile generation. Our experiments demonstrate that models like Mistral-7b and Llama2-7b perform strongly in both tasks. LLMs improve the precision and recall of the generated profiles, and high evaluation scores confirm the effectiveness of our approach.