Breaking the Top-$K$ Barrier: Advancing Top-$K$ Ranking Metrics Optimization in Recommender Systems

TL;DR

This paper introduces SL@$K$, a new loss function for recommender systems that effectively optimizes Top-$K$ ranking metrics like NDCG@$K$, overcoming previous challenges related to discontinuity and computational cost.

Contribution

We propose SL@$K$, a novel smooth loss function that directly optimizes NDCG@$K$ by handling Top-$K$ truncation and discontinuity, with theoretical guarantees and practical efficiency.

Findings

SL@$K$ outperforms existing loss functions by an average of 6.03% in experiments.

The proposed loss is computationally efficient and stable during training.

Extensive experiments on multiple datasets validate the effectiveness of SL@$K$.

Abstract

In the realm of recommender systems (RS), Top-$K$ ranking metrics such as NDCG@$K$ are the gold standard for evaluating recommendation performance. However, during the training of recommendation models, optimizing NDCG@$K$ poses significant challenges due to its inherent discontinuous nature and the intricate Top-$K$ truncation. Recent efforts to optimize NDCG@$K$ have either overlooked the Top-$K$ truncation or suffered from high computational costs and training instability. To overcome these limitations, we propose SoftmaxLoss@$K$ (SL@$K$), a novel recommendation loss tailored for NDCG@$K$ optimization. Specifically, we integrate the quantile technique to handle Top-$K$ truncation and derive a smooth upper bound for optimizing NDCG@$K$ to address discontinuity. The resulting SL@$K$ loss has several desirable properties, including theoretical guarantees, ease of implementation, computational efficiency, gradient stability, and noise robustness. Extensive experiments on four real-world datasets and three recommendation backbones demonstrate that SL@$K$ outperforms existing losses with a notable average improvement of 6.03%. The code is available at https://github.com/Tiny-Snow/IR-Benchmark.