Bending the Scaling Law Curve in Large-Scale Recommendation Systems

TL;DR

This paper introduces ULTRA-HSTU, a novel recommendation model that significantly improves training and inference efficiency while enhancing recommendation quality through innovative sequence design and sparse attention mechanisms, deployed at scale.

Contribution

The paper presents ULTRA-HSTU, a new sequential recommendation model with end-to-end co-design, achieving over 5x faster training, 21x faster inference, and superior recommendation performance.

Findings

Over 5x faster training scaling

21x faster inference scaling

4-8% improvements in user engagement

Abstract

Learning from user interaction history through sequential models has become a cornerstone of large-scale recommender systems. Recent advances in large language models have revealed promising scaling laws, sparking a surge of research into long-sequence modeling and deeper architectures for recommendation tasks. However, many recent approaches rely heavily on cross-attention mechanisms to address the quadratic computational bottleneck in sequential modeling, which can limit the representational power gained from self-attention. We present ULTRA-HSTU, a novel sequential recommendation model developed through end-to-end model and system co-design. By innovating in the design of input sequences, sparse attention mechanisms, and model topology, ULTRA-HSTU achieves substantial improvements in both model quality and efficiency. Comprehensive benchmarking demonstrates that ULTRA-HSTU achieves remarkable scaling efficiency gains -- over 5x faster training scaling and 21x faster inference scaling compared to conventional models -- while delivering superior recommendation quality. Our solution is fully deployed at scale, serving billions of users daily and driving significant 4% to 8% consumption and engagement improvements in real-world production environments.