Enhancing LambdaMART Using Oblivious Trees

TL;DR

This paper shows that replacing standard regression trees with oblivious trees in LambdaMART improves document retrieval performance by over 2.2%, supported by experimental comparisons and analysis.

Contribution

It introduces the use of oblivious trees in LambdaMART, demonstrating performance gains over standard trees in ranking tasks.

Findings

Oblivious trees improve LambdaMART performance by more than 2.2%.

Experimental analysis confirms the benefits of oblivious decision trees.

Performance gains are consistent across different feature sets and training sizes.

Abstract

Learning to rank is a machine learning technique broadly used in many areas such as document retrieval, collaborative filtering or question answering. We present experimental results which suggest that the performance of the current state-of-the-art learning to rank algorithm LambdaMART, when used for document retrieval for search engines, can be improved if standard regression trees are replaced by oblivious trees. This paper provides a comparison of both variants and our results demonstrate that the use of oblivious trees can improve the performance by more than $2.2\%$. Additional experimental analysis of the influence of a number of features and of a size of the training set is also provided and confirms the desirability of properties of oblivious decision trees.