Make It Long, Keep It Fast: End-to-End 10K Long User Behavior Sequence Modeling for Billion-Scale Douyin Recommendation

TL;DR

This paper introduces a scalable, end-to-end recommendation system for Douyin that effectively models 10,000-length user behavior sequences using novel attention and batching techniques, achieving significant online performance improvements.

Contribution

It presents the Stacked Target-to-History Cross Attention (STCA), Request Level Batching (RLB), and a length-extrapolative training strategy for efficient ultra-long sequence modeling in production.

Findings

Scaling to 10K-length histories improves engagement metrics.

The proposed methods reduce computational complexity and storage.

The system maintains low latency while handling ultra-long sequences.

Abstract

Short-video recommenders such as Douyin must exploit extremely long user behavior histories without breaking latency or cost budgets. We present an end-to-end industrial recommender system that scales long-sequence recommendation modeling to 10K-length histories in production. First, we introduce Stacked Target-to-History Cross Attention (STCA), which replaces history self-attention with stacked cross-attention from the target to the history, reducing complexity from quadratic to linear in sequence length and enabling efficient end-to-end training over long user behavior sequences. Second, we propose Request Level Batching (RLB), a user-centric batching scheme that aggregates multiple targets for the same user/request to share the user-side encoding, substantially lowering sequence-related storage, communication, and compute without changing the learning objective. Third, we design a length-extrapolative training strategy -- train on shorter windows, infer on much longer ones -- so the model generalizes to 10K-scale histories without additional training cost. Across offline and online experiments, we observe predictable, monotonic gains as we scale history length and model capacity, mirroring the scaling law behavior observed in large language models. Deployed at full traffic on Douyin, our system delivers significant improvements on key engagement metrics while meeting production latency, demonstrating a practical path to scaling end-to-end ultra-long sequence recommendation to the 10K regime.