Multi-Margin Cosine Loss: Proposal and Application in Recommender Systems

TL;DR

This paper introduces Multi-Margin Cosine Loss (MMCL), a novel loss function for recommender systems that improves performance by effectively utilizing negative samples, especially under resource constraints.

Contribution

The paper proposes MMCL, a new loss function with multiple margins and weights that enhances negative sample utilization and outperforms existing methods in recommender systems.

Findings

MMCL achieves up to 20% performance improvement with fewer negatives.

It effectively utilizes a broader range of negative samples.

The method simplifies contrastive learning in resource-limited settings.

Abstract

Recommender systems guide users through vast amounts of information by suggesting items based on their predicted preferences. Collaborative filtering-based deep learning techniques have regained popularity due to their straightforward nature, relying only on user-item interactions. Typically, these systems consist of three main components: an interaction module, a loss function, and a negative sampling strategy. Initially, researchers focused on enhancing performance by developing complex interaction modules. However, there has been a recent shift toward refining loss functions and negative sampling strategies. This shift has led to an increased interest in contrastive learning, which pulls similar pairs closer while pushing dissimilar ones apart. Contrastive learning may bring challenges like high memory demands and under-utilization of some negative samples. The proposed Multi-Margin Cosine Loss (MMCL) addresses these challenges by introducing multiple margins and varying weights for negative samples. It efficiently utilizes not only the hardest negatives but also other non-trivial negatives, offers a simpler yet effective loss function that outperforms more complex methods, especially when resources are limited. Experiments on two well-known datasets demonstrated that MMCL achieved up to a 20\% performance improvement compared to a baseline loss function when fewer number of negative samples are used.