Text as a Universal Interface for Transferable Personalization

TL;DR

This paper proposes using natural language as a universal, interpretable interface for personalization in large language models, enabling transferability and continual evolution of user preferences.

Contribution

It introduces a two-stage training framework and the AlignXplore+ model that generate textual preference summaries for improved transferability and interpretability.

Findings

Achieves state-of-the-art performance on nine benchmarks.

Outperforms larger models in transferability and utility.

Demonstrates strong cross-task and cross-model generalization.

Abstract

We study the problem of personalization in large language models (LLMs). Prior work predominantly represents user preferences as implicit, model-specific vectors or parameters, yielding opaque ``black-box'' profiles that are difficult to interpret and transfer across models and tasks. In contrast, we advocate natural language as a universal, model- and task-agnostic interface for preference representation. The formulation leads to interpretable and reusable preference descriptions, while naturally supporting continual evolution as new interactions are observed. To learn such representations, we introduce a two-stage training framework that combines supervised fine-tuning on high-quality synthesized data with reinforcement learning to optimize long-term utility and cross-task transferability. Based on this framework, we develop AlignXplore+, a universal preference reasoning model that generates textual preference summaries. Experiments on nine benchmarks show that our 8B model achieves state-of-the-art performanc -- outperforming substantially larger open-source models -- while exhibiting strong transferability across tasks, model families, and interaction formats.