Rethinking Personalization in Large Language Models at the Token Level

TL;DR

This paper introduces PerContrast and PerCE, a novel token-level personalization method for large language models that improves personalized output quality by estimating and emphasizing user-specific tokens during training.

Contribution

It proposes a new token-level personalization framework using causal intervention and a bootstrap-based loss, enhancing personalization in LLMs with minimal extra cost.

Findings

Over 10% average improvement in personalization performance.

Up to 68.04% improvement on LongLaMP dataset.

Strong transferability across tasks and scenarios.

Abstract

With large language models (LLMs) now performing strongly across diverse tasks, there is growing demand for them to personalize outputs for individual users. Personalization is typically framed as an additional layer on top of a base NLP task, requiring model responses to meet user-specific needs while still accomplishing the underlying task. From a token-level perspective, different tokens in a response contribute to personalization to varying degrees. Tokens with higher personalization relevance should therefore receive greater emphasis when developing personalized LLMs. However, accurately estimating such personalization degrees remains challenging. To address this challenge, we propose PerContrast, a self-contrast method that estimates each output token's dependence on user-specific information through causal intervention. Building on this mechanism, we develop the PerCE loss, which adaptively upweights tokens with higher estimated personalization degrees during training via a bootstrap procedure, enabling the model to alternate between estimating and optimizing these tokens. Experiments on multiple LLMs demonstrate that PerCE substantially improves personalization performance with minimal additional cost, achieving average gains of over 10% and up to 68.04% on the LongLaMP dataset, along with strong cross-task and cross-scenario transferability. These results highlight the importance of token-level personalization modeling and establish token-aware training as a simple yet effective paradigm for advancing personalized LLMs.