IAT: Instance-As-Token Compression for Historical User Sequence Modeling in Industrial Recommender Systems

TL;DR

The paper introduces IAT, a two-stage sequence modeling framework that compresses user interaction data into tokens for improved recommendation performance and transferability.

Contribution

It proposes a novel Instance-As-Token approach with temporal and user-order compression schemes, enhancing sequence modeling in industrial recommender systems.

Findings

IAT outperforms state-of-the-art methods in experiments.

It improves key business metrics in real-world deployments.

IAT demonstrates strong transferability across domains.

Abstract

Although sophisticated sequence modeling paradigms have achieved remarkable success in recommender systems, the information capacity of hand-crafted sequential features constrains the performance upper bound. To better enhance user experience by encoding historical interaction patterns, this paper presents a novel two-stage sequence modeling framework termed Instance-As-Token (IAT). The first stage of IAT compresses all features of each historical interaction instance into a unified instance embedding, which encodes the interaction characteristics in a compact yet informative token. Both temporal-order and user-order compression schemes are proposed, with the latter better aligning with the demands of downstream sequence modeling. The second stage involves the downstream task fetching fixed-length compressed instance tokens via timestamps and adopting standard sequence modeling approaches to learn long-range preferences patterns. Extensive experiments demonstrate that IAT significantly outperforms state-of-the-art methods and exhibits superior in-domain and cross-domain transferability. IAT has been successfully deployed in real-world industrial recommender systems, including e-commerce advertising, shopping mall marketing, and live-streaming e-commerce, delivering substantial improvements in key business metrics.