AutoLoss: Automated Loss Function Search in Recommendations

TL;DR

AutoLoss introduces an adaptive, differentiable controller to automatically search and adjust loss functions for deep recommender systems, enhancing performance and efficiency across datasets.

Contribution

The paper presents a novel AutoLoss framework with a dynamic controller that adaptively generates loss probabilities based on data convergence behaviors, improving over fixed or manually fused loss functions.

Findings

AutoLoss outperforms baseline methods on benchmark datasets.

The framework demonstrates high transferability across different datasets.

AutoLoss improves training efficiency and model generalizability.

Abstract

Designing an effective loss function plays a crucial role in training deep recommender systems. Most existing works often leverage a predefined and fixed loss function that could lead to suboptimal recommendation quality and training efficiency. Some recent efforts rely on exhaustively or manually searched weights to fuse a group of candidate loss functions, which is exceptionally costly in computation and time. They also neglect the various convergence behaviors of different data examples. In this work, we propose an AutoLoss framework that can automatically and adaptively search for the appropriate loss function from a set of candidates. To be specific, we develop a novel controller network, which can dynamically adjust the loss probabilities in a differentiable manner. Unlike existing algorithms, the proposed controller can adaptively generate the loss probabilities for different data examples according to their varied convergence behaviors. Such design improves the model's generalizability and transferability between deep recommender systems and datasets. We evaluate the proposed framework on two benchmark datasets. The results show that AutoLoss outperforms representative baselines. Further experiments have been conducted to deepen our understandings of AutoLoss, including its transferability, components and training efficiency.