Empowering General-purpose User Representation with Full-life Cycle Behavior Modeling

TL;DR

This paper introduces LURM, a novel framework for full-life cycle user behavior modeling that encodes long-term user interests into general-purpose representations, outperforming existing methods on benchmark datasets.

Contribution

The paper proposes a new full-life cycle user representation framework with a two-stage model, enabling effective modeling of extremely long behavior sequences.

Findings

LURM outperforms state-of-the-art methods on benchmark datasets.

The multi-anchor encoder effectively captures diverse user interests.

The approach achieves near lossless dimensionality reduction.

Abstract

User Modeling plays an essential role in industry. In this field, task-agnostic approaches, which generate general-purpose representation applicable to diverse downstream user cognition tasks, is a promising direction being more valuable and economical than task-specific representation learning. With the rapid development of Internet service platforms, user behaviors have been accumulated continuously. However, existing general-purpose user representation researches have little ability for full-life cycle modeling on extremely long behavior sequences since user registration. In this study, we propose a novel framework called full- Life cycle User Representation Model (LURM) to tackle this challenge. Specifically, LURM consists of two cascaded sub-models: (I) Bag-of-Interests (BoI) encodes user behaviors in any time period into a sparse vector with super-high dimension (e.g., 10^5); (II) Self-supervised Multi-anchor Encoder Network (SMEN) maps sequences of BoI features to multiple low-dimensional user representations. Specially, SMEN achieves almost lossless dimensionality reduction, benefiting from a novel multi-anchor module which can learn different aspects of user interests. Experiments on several benchmark datasets show that our approach outperforms state-of-the-art general-purpose representation methods.