Metric@CustomerN: Evaluating Metrics at a Customer Level in E-Commerce

TL;DR

This paper proposes using median impressions per diner as a personalized Top-N metric for evaluating recommendation systems in e-commerce, moving beyond fixed Top-N measures.

Contribution

It introduces a personalized evaluation approach based on median impressions, offering a new perspective for recommendation system assessment.

Findings

Preliminary results suggest potential benefits of personalized metrics.

Future work includes refining and validating the personalized evaluation method.

Abstract

Accuracy measures such as Recall, Precision, and Hit Rate have been a standard way of evaluating Recommendation Systems. The assumption is to use a fixed Top-N to represent them. We propose that median impressions viewed from historical sessions per diner be used as a personalized value for N. We present preliminary exploratory results and list future steps to improve upon and evaluate the efficacy of these personalized metrics.