Scalable Cross-Entropy Loss for Sequential Recommendations with Large Item Catalogs

TL;DR

This paper proposes a scalable Cross-Entropy loss function for sequential recommendation systems with large item catalogs, significantly reducing memory usage while maintaining or improving recommendation quality.

Contribution

It introduces a novel SCE loss that approximates traditional CE efficiently, enabling large-scale recommendations without high GPU memory consumption.

Findings

Reduces peak memory usage by up to 100 times

Maintains or exceeds recommendation quality metrics

Effective for large-scale datasets and models

Abstract

Scalability issue plays a crucial role in productionizing modern recommender systems. Even lightweight architectures may suffer from high computational overload due to intermediate calculations, limiting their practicality in real-world applications. Specifically, applying full Cross-Entropy (CE) loss often yields state-of-the-art performance in terms of recommendations quality. Still, it suffers from excessive GPU memory utilization when dealing with large item catalogs. This paper introduces a novel Scalable Cross-Entropy (SCE) loss function in the sequential learning setup. It approximates the CE loss for datasets with large-size catalogs, enhancing both time efficiency and memory usage without compromising recommendations quality. Unlike traditional negative sampling methods, our approach utilizes a selective GPU-efficient computation strategy, focusing on the most informative elements of the catalog, particularly those most likely to be false positives. This is achieved by approximating the softmax distribution over a subset of the model outputs through the maximum inner product search. Experimental results on multiple datasets demonstrate the effectiveness of SCE in reducing peak memory usage by a factor of up to 100 compared to the alternatives, retaining or even exceeding their metrics values. The proposed approach also opens new perspectives for large-scale developments in different domains, such as large language models.