Clicks Versus Conversion: Choosing a Recommender's Training Objective in E-Commerce

TL;DR

This paper compares optimizing recommender systems for click-through rate versus conversion metrics in e-commerce, finding that focusing on conversion significantly boosts gross merchandise value without harming product discovery.

Contribution

It provides an empirical comparison of CTR and conversion-based objectives in recommender training, highlighting the superior impact of conversion optimization on business outcomes.

Findings

Optimizing for OSR yields over five times GMV uplift compared to CTR.

Conversion objectives do not reduce new product discovery.

Different feature importances are observed for each objective.

Abstract

Ranking product recommendations to optimize for a high click-through rate (CTR) or for high conversion, such as add-to-cart rate (ACR) and Order-Submit-Rate (OSR, view-to-purchase conversion) are standard practices in e-commerce. Optimizing for CTR appears like a straightforward choice: Training data (i.e., click data) are simple to collect and often available in large quantities. Additionally, CTR is used far beyond e-commerce, making it a generalist, easily implemented option. ACR and OSR, on the other hand, are more directly linked to a shop's business goals, such as the Gross Merchandise Value (GMV). In this paper, we compare the effects of using either of these objectives using an online A/B test. Among our key findings, we demonstrate that in our shops, optimizing for OSR produces a GMV uplift more than five times larger than when optimizing for CTR, without sacrificing new product discovery. Our results also provide insights into the different feature importances for each of the objectives.