Which Tricks Are Important for Learning to Rank?

TL;DR

This paper compares various gradient-boosted decision tree methods for learning-to-rank, analyzing their effectiveness and proposing improvements to optimize ranking loss functions, leading to a new state-of-the-art algorithm.

Contribution

It provides a unified analysis of GBDT-based ranking algorithms, compares direct loss optimization with surrogate methods, and introduces a simple yet effective enhancement for YetiRank.

Findings

Direct optimization of smoothed ranking loss can outperform surrogate methods.

The proposed YetiRank modification improves ranking performance.

The new algorithm achieves state-of-the-art results in learning-to-rank tasks.

Abstract

Nowadays, state-of-the-art learning-to-rank methods are based on gradient-boosted decision trees (GBDT). The most well-known algorithm is LambdaMART which was proposed more than a decade ago. Recently, several other GBDT-based ranking algorithms were proposed. In this paper, we thoroughly analyze these methods in a unified setup. In particular, we address the following questions. Is direct optimization of a smoothed ranking loss preferable over optimizing a convex surrogate? How to properly construct and smooth surrogate ranking losses? To address these questions, we compare LambdaMART with YetiRank and StochasticRank methods and their modifications. We also propose a simple improvement of the YetiRank approach that allows for optimizing specific ranking loss functions. As a result, we gain insights into learning-to-rank techniques and obtain a new state-of-the-art algorithm.