Mathematical Modeling of Product Rating: Sufficiency, Misbehavior and Aggregation Rules

TL;DR

This paper develops a mathematical model to determine the minimum ratings needed for reliable product evaluation under partial information, accounting for user misbehavior, and shows the superiority of majority rule over average in aggregation.

Contribution

It introduces a formal model for product rating evaluation with partial data, providing bounds and robustness analysis against misbehavior, validated by real-world data.

Findings

Majority rating rule outperforms average rating in reliability.

Theoretical bounds on minimum ratings for reliable evaluation.

Model accounts for user misbehavior affecting ratings.

Abstract

Many web services like eBay, Tripadvisor, Epinions, etc, provide historical product ratings so that users can evaluate the quality of products. Product ratings are important since they affect how well a product will be adopted by the market. The challenge is that we only have {\em "partial information"} on these ratings: Each user provides ratings to only a "{\em small subset of products}". Under this partial information setting, we explore a number of fundamental questions: What is the "{\em minimum number of ratings}" a product needs so one can make a reliable evaluation of its quality? How users' {\em misbehavior} (such as {\em cheating}) in product rating may affect the evaluation result? To answer these questions, we present a formal mathematical model of product evaluation based on partial information. We derive theoretical bounds on the minimum number of ratings needed to produce a reliable indicator of a product's quality. We also extend our model to accommodate users' misbehavior in product rating. We carry out experiments using both synthetic and real-world data (from TripAdvisor, Amazon and eBay) to validate our model, and also show that using the "majority rating rule" to aggregate product ratings, it produces more reliable and robust product evaluation results than the "average rating rule".