CURP: Codebook-based Continuous User Representation for Personalized Generation with LLMs

TL;DR

CURP introduces a codebook-based user modeling framework that enhances personalized generation with LLMs, achieving high performance, interpretability, and efficiency with minimal additional parameters.

Contribution

The paper presents a novel plug-and-play user representation method using a discrete codebook, improving personalization quality and scalability in LLM-based generation.

Findings

Outperforms strong baselines in variant generation tasks

Uses only about 20M trainable parameters for personalization

Offers better interpretability and scalability

Abstract

User modeling characterizes individuals through their preferences and behavioral patterns to enable personalized simulation and generation with Large Language Models (LLMs) in contemporary approaches. However, existing methods, whether prompt-based or training-based methods, face challenges in balancing personalization quality against computational and data efficiency. We propose a novel framework CURP, which employs a bidirectional user encoder and a discrete prototype codebook to extract multi-dimensional user traits. This design enables plug-and-play personalization with a small number of trainable parameters (about 20M parameters, about 0.2\% of the total model size). Through extensive experiments on variant generation tasks, we show that CURP achieves superior performance and generalization compared to strong baselines, while offering better interpretability and scalability. The code are available at https://github.com/RaidonWong/CURP_code