Learning to Comparison-Shop

TL;DR

This paper introduces the LTCS system, a novel ranking model that explicitly captures user comparison shopping behavior, leading to improved relevance, diversity, and user satisfaction in online marketplaces.

Contribution

The paper presents a new ranking architecture that models comparison shopping behavior, outperforming existing methods in relevance and user experience metrics.

Findings

NDCG improved by 1.7%

Booking conversion rate increased by 0.6%

Outperforms state-of-the-art models

Abstract

In online marketplaces like Airbnb, users frequently engage in comparison shopping before making purchase decisions. Despite the prevalence of this behavior, a significant disconnect persists between mainstream e-commerce search engines and users' comparison needs. Traditional ranking models often evaluate items in isolation, disregarding the context in which users compare multiple items on a search results page. While recent advances in deep learning have sought to improve ranking accuracy, diversity, and fairness by encoding listwise context, the challenge of aligning search rankings with user comparison shopping behavior remains inadequately addressed. In this paper, we propose a novel ranking architecture - Learning-to-Comparison-Shop (LTCS) System - that explicitly models and learns users' comparison shopping behaviors. Through extensive offline and online experiments, we demonstrate that our approach yields statistically significant gains in key business metrics - improving NDCG by 1.7% and boosting booking conversion rate by 0.6% in A/B testing - while also enhancing user experience. We also compare our model against state-of-the-art approaches and demonstrate that LTCS significantly outperforms them.