Learning To Rank Diversely At Airbnb

TL;DR

This paper challenges the assumption that booking probabilities are independent in ranking models, introduces a theoretical correction, and demonstrates how diversifying results improves large-scale Airbnb search performance.

Contribution

It provides a theoretical foundation and neural network architectures that account for listing similarities to enhance diversification in ranking systems.

Findings

Diversification improves user engagement and satisfaction.

Theoretical correction enhances ranking accuracy.

Online A/B tests show significant performance gains.

Abstract

Airbnb is a two-sided marketplace, bringing together hosts who own listings for rent, with prospective guests from around the globe. Applying neural network-based learning to rank techniques has led to significant improvements in matching guests with hosts. These improvements in ranking were driven by a core strategy: order the listings by their estimated booking probabilities, then iterate on techniques to make these booking probability estimates more and more accurate. Embedded implicitly in this strategy was an assumption that the booking probability of a listing could be determined independently of other listings in search results. In this paper we discuss how this assumption, pervasive throughout the commonly-used learning to rank frameworks, is false. We provide a theoretical foundation correcting this assumption, followed by efficient neural network architectures based on the theory. Explicitly accounting for possible similarities between listings, and reducing them to diversify the search results generated strong positive impact. We discuss these metric wins as part of the online A/B tests of the theory. Our method provides a practical way to diversify search results for large-scale production ranking systems.