User Long-Term Multi-Interest Retrieval Model for Recommendation

TL;DR

This paper introduces ULIM, a novel retrieval model that effectively captures long-term user interests across thousands of behaviors, significantly improving recommendation performance in large-scale industrial systems.

Contribution

ULIM is the first retrieval framework to model thousand-scale user behavior sequences by integrating category-aware interest partitioning and a pointer-enhanced cascaded retrieval architecture.

Findings

ULIM outperforms state-of-the-art methods on Taobao dataset.

ULIM achieves 5.54% increase in clicks.

ULIM results in 11.01% increase in orders and 4.03% GMV lift.

Abstract

User behavior sequence modeling, which captures user interest from rich historical interactions, is pivotal for industrial recommendation systems. Despite breakthroughs in ranking-stage models capable of leveraging ultra-long behavior sequences with length scaling up to thousands, existing retrieval models remain constrained to sequences of hundreds of behaviors due to two main challenges. One is strict latency budget imposed by real-time service over large-scale candidate pool. The other is the absence of target-aware mechanisms and cross-interaction architectures, which prevent utilizing ranking-like techniques to simplify long sequence modeling. To address these limitations, we propose a new framework named User Long-term Multi-Interest Retrieval Model(ULIM), which enables thousand-scale behavior modeling in retrieval stages. ULIM includes two novel components: 1)Category-Aware Hierarchical Dual-Interest Learning partitions long behavior sequences into multiple category-aware subsequences representing multi-interest and jointly optimizes long-term and short-term interests within specific interest cluster. 2)Pointer-Enhanced Cascaded Category-to-Item Retrieval introduces Pointer-Generator Interest Network(PGIN) for next-category prediction, followed by next-item retrieval upon the top-K predicted categories. Comprehensive experiments on Taobao dataset show that ULIM achieves substantial improvement over state-of-the-art methods, and brings 5.54% clicks, 11.01% orders and 4.03% GMV lift for Taobaomiaosha, a notable mini-app of Taobao.