Analysis of Regression Tree Fitting Algorithms in Learning to Rank

TL;DR

This paper introduces a new tree fitting principle called least objective loss based error for learning to rank, analyzing its relationship with existing methods and demonstrating moderate improvements through experiments.

Contribution

It proposes a novel tree fitting principle for learning to rank and provides analysis and empirical validation of its effectiveness.

Findings

Moderate performance improvements over traditional least square error.

Analysis of the relationship between different tree fitting principles.

Implementation of two strong learning to rank systems.

Abstract

In learning to rank area, industry-level applications have been dominated by gradient boosting framework, which fits a tree using least square error principle. While in classification area, another tree fitting principle, weighted least square error, has been widely used, such as LogitBoost and its variants. However, there is a lack of analysis on the relationship between the two principles in the scenario of learning to rank. We propose a new principle named least objective loss based error that enables us to analyze the issue above as well as several important learning to rank models. We also implement two typical and strong systems and conduct our experiments in two real-world datasets. Experimental results show that our proposed method brings moderate improvements over least square error principle.