Reinforced Prompt Personalization for Recommendation with Large Language Models

TL;DR

This paper introduces Reinforced Prompt Personalization (RPP), a novel method for customizing prompts at the instance level for individual users in recommendation systems using large language models, improving accuracy and scalability.

Contribution

It proposes a new instance-wise prompt personalization approach, RPP, that dynamically refines prompts at the sentence level, enhancing recommendation performance over traditional task-wise prompting methods.

Findings

RPP outperforms traditional recommender models and few-shot methods.

RPP+ further refines actions for improved scalability and accuracy.

Experiments demonstrate the effectiveness of personalized prompts in recommendation tasks.

Abstract

Designing effective prompts can empower LLMs to understand user preferences and provide recommendations with intent comprehension and knowledge utilization capabilities. Nevertheless, recent studies predominantly concentrate on task-wise prompting, developing fixed prompt templates shared across all users in a given recommendation task (e.g., rating or ranking). Although convenient, task-wise prompting overlooks individual user differences, leading to inaccurate analysis of user interests. In this work, we introduce the concept of instance-wise prompting, aiming at personalizing discrete prompts for individual users. Toward this end, we propose Reinforced Prompt Personalization (RPP) to realize it automatically. To improve efficiency and quality, RPP personalizes prompts at the sentence level rather than searching in the vast vocabulary word-by-word. Specifically, RPP breaks down the prompt into four patterns, tailoring patterns based on multi-agent and combining them. Then the personalized prompts interact with LLMs (environment) iteratively, to boost LLMs' recommending performance (reward). In addition to RPP, to improve the scalability of action space, our proposal of RPP+ dynamically refines the selected actions with LLMs throughout the iterative process. Extensive experiments on various datasets demonstrate the superiority of RPP/RPP+ over traditional recommender models, few-shot methods, and other prompt-based methods, underscoring the significance of instance-wise prompting in LLMs for recommendation. Our code is available at https://github.com/maowenyu-11/RPP.