Understanding the Role of User Profile in the Personalization of Large Language Models

TL;DR

This paper investigates how user profiles influence the personalization of Large Language Models, revealing that historical responses are key and placement within input affects effectiveness, providing insights for better personalization strategies.

Contribution

It clarifies the role of user profiles in LLM personalization, emphasizing the importance of historical responses and input positioning, which was previously unclear.

Findings

Historical personalized responses are crucial for personalization.

User profile placement closer to input start has greater impact.

Personalization effectiveness is mainly due to personalization info, not semantic info.

Abstract

Utilizing user profiles to personalize Large Language Models (LLMs) has been shown to enhance the performance on a wide range of tasks. However, the precise role of user profiles and their effect mechanism on LLMs remains unclear. This study first confirms that the effectiveness of user profiles is primarily due to personalization information rather than semantic information. Furthermore, we investigate how user profiles affect the personalization of LLMs. Within the user profile, we reveal that it is the historical personalized response produced or approved by users that plays a pivotal role in personalizing LLMs. This discovery unlocks the potential of LLMs to incorporate a greater number of user profiles within the constraints of limited input length. As for the position of user profiles, we observe that user profiles integrated into different positions of the input context do not contribute equally to personalization. Instead, where the user profile that is closer to the beginning affects more on the personalization of LLMs. Our findings reveal the role of user profiles for the personalization of LLMs, and showcase how incorporating user profiles impacts performance providing insight to leverage user profiles effectively.