Data Preprocessing for Evaluation of Recommendation Models in E-Commerce

TL;DR

This paper presents a comprehensive data preprocessing methodology for evaluating recommendation models in e-commerce, addressing issues from customer behavior to user interface to improve accuracy in measuring recommendation impact.

Contribution

It introduces novel strategies for outlier removal, data collection, and UI analysis to enhance the validity of recommendation evaluation metrics.

Findings

Improved accuracy in measuring recommendation influence on customer actions.

Effective outlier removal techniques for e-commerce data.

Robust methods for tracking visitors and attributing purchases.

Abstract

E-commerce businesses employ recommender models to assist in identifying a personalized set of products for each visitor. To accurately assess the recommendations' influence on customer clicks and buys, three target areas -- customer behavior, data collection, user-interface -- will be explored for possible sources of erroneous data. Varied customer behavior misrepresents the recommendations' true influence on a customer due to the presence of B2B interactions and outlier customers. Non-parametric statistical procedures for outlier removal are delineated and other strategies are investigated to account for the effect of a large percentage of new customers or high bounce rates. Subsequently, in data collection we identify probable misleading interactions in the raw data, propose a robust method of tracking unique visitors, and accurately attributing the buy influence for combo products. Lastly, user-interface issues discuss the possible problems caused due to the recommendation widget's positioning on the e-commerce website and the stringent conditions that should be imposed when utilizing data from the product listing page. This collective methodology results in an exact and valid estimation of the customer's interactions influenced by the recommendation model in the context of standard industry metrics, such as Click-through rates, Buy-through rates, and Conversion revenue.