Query as Anchor: Scenario-Adaptive User Representation via Large Language Model

TL;DR

This paper introduces Query-as-Anchor, a dynamic user representation framework using large language models that adapts to specific scenarios, improves task sensitivity, and handles multi-source data noise, validated through industrial benchmarks and online testing.

Contribution

It presents a novel query-aware user modeling approach with a new pre-training dataset, hierarchical encoders, and soft prompt tuning, advancing beyond static embeddings for industrial applications.

Findings

Achieved state-of-the-art performance on 10 industrial benchmarks.

Demonstrated strong scalability and deployment efficiency.

Validated effectiveness through large-scale online A/B testing.

Abstract

Industrial-scale user representation learning requires balancing robust universality with acute task-sensitivity. However, existing paradigms primarily yield static, task-agnostic embeddings that struggle to reconcile the divergent requirements of downstream scenarios within unified vector spaces. Furthermore, heterogeneous multi-source data introduces inherent noise and modality conflicts, degrading representation. We propose Query-as-Anchor, a framework shifting user modeling from static encoding to dynamic, query-aware synthesis. To empower Large Language Models (LLMs) with deep user understanding, we first construct UserU, an industrial-scale pre-training dataset that aligns multi-modal behavioral sequences with user understanding semantics, and our Q-Anchor Embedding architecture integrates hierarchical coarse-to-fine encoders into dual-tower LLMs via joint contrastive-autoregressive optimization for query-aware user representation. To bridge the gap between general pre-training and specialized business logic, we further introduce Cluster-based Soft Prompt Tuning to enforce discriminative latent structures, effectively aligning model attention with scenario-specific modalities. For deployment, anchoring queries at sequence termini enables KV-cache-accelerated inference with negligible incremental latency. Evaluations on 10 Alipay industrial benchmarks show consistent SOTA performance, strong scalability, and efficient deployment. Large-scale online A/B testing in Alipay's production system across two real-world scenarios further validates its practical effectiveness. Our code is prepared for public release and will be available at: https://github.com/JhCircle/Q-Anchor.