ORDSIM: Ordinal Regression for E-Commerce Query Similarity Prediction

TL;DR

This paper introduces ORDSIM, an ordinal regression model designed to improve high-similarity query predictions in e-commerce, outperforming traditional regression models by focusing more on high-similarity errors.

Contribution

The paper proposes a novel ordinal regression approach with variable-width buckets to better predict high-similarity queries in e-commerce search.

Findings

ORDSIM achieves lower prediction error than regression baselines.

The model effectively emphasizes high-similarity predictions.

Evaluation on 10 million queries demonstrates its practical advantage.

Abstract

Query similarity prediction task is generally solved by regression based models with square loss. Such a model is agnostic of absolute similarity values and it penalizes the regression error at all ranges of similarity values at the same scale. However, to boost e-commerce platform's monetization, it is important to predict high-level similarity more accurately than low-level similarity, as highly similar queries retrieves items according to user-intents, whereas moderately similar item retrieves related items, which may not lead to a purchase. Regression models fail to customize its loss function to concentrate around the high-similarity band, resulting poor performance in query similarity prediction task. We address the above challenge by considering the query prediction as an ordinal regression problem, and thereby propose a model, ORDSIM (ORDinal Regression for SIMilarity Prediction). ORDSIM exploits variable-width buckets to model ordinal loss, which penalizes errors in high-level similarity harshly, and thus enable the regression model to obtain better prediction results for high similarity values. We evaluate ORDSIM on a dataset of over 10 millions e-commerce queries from eBay platform and show that ORDSIM achieves substantially smaller prediction error compared to the competing regression methods on this dataset.