Advancing Loss Functions in Recommender Systems: A Comparative Study with a R\'enyi Divergence-Based Solution

TL;DR

This paper analyzes popular loss functions in recommender systems, revealing their strengths and limitations, and introduces a new Rényi-divergence-based loss function, DrRL, that outperforms existing methods in accuracy and robustness.

Contribution

The paper provides a comprehensive analysis of Softmax Loss and Cosine Contrastive Loss, and proposes a novel Rényi-divergence-based loss function, DrRL, to improve recommendation robustness and accuracy.

Findings

DrRL outperforms existing loss functions in recommendation accuracy.

DrRL enhances robustness against distributional shifts.

Analysis reveals strengths and limitations of SL and CCL.

Abstract

Loss functions play a pivotal role in optimizing recommendation models. Among various loss functions, Softmax Loss (SL) and Cosine Contrastive Loss (CCL) are particularly effective. Their theoretical connections and differences warrant in-depth exploration. This work conducts comprehensive analyses of these losses, yielding significant insights: 1) Common strengths -- both can be viewed as augmentations of traditional losses with Distributional Robust Optimization (DRO), enhancing robustness to distributional shifts; 2) Respective limitations -- stemming from their use of different distribution distance metrics in DRO optimization, SL exhibits high sensitivity to false negative instances, whereas CCL suffers from low data utilization. To address these limitations, this work proposes a new loss function, DrRL, which generalizes SL and CCL by leveraging R\'enyi-divergence in DRO optimization. DrRL incorporates the advantageous structures of both SL and CCL, and can be demonstrated to effectively mitigate their limitations. Extensive experiments have been conducted to validate the superiority of DrRL on both recommendation accuracy and robustness.