Eliminating the effect of rating bias on reputation systems

TL;DR

This paper introduces an iterative balance method to correct rating biases in reputation systems, improving the accuracy of object quality evaluation and user reputation assessment on online platforms.

Contribution

The paper proposes a novel iterative balance (IB) method that effectively eliminates rating bias effects, enhancing reputation system reliability and object quality estimation.

Findings

IB method accurately quantifies movie quality and user rating stability.

IB outperforms existing methods in identifying underrated but high-quality movies.

Experiments on MovieLens and Netflix datasets demonstrate robustness and self-consistency.

Abstract

The ongoing rapid development of the e-commercial and interest-base websites make it more pressing to evaluate objects' accurate quality before recommendation by employing an effective reputation system. The objects' quality are often calculated based on their historical information, such as selected records or rating scores, to help visitors to make decisions before watching, reading or buying. Usually high quality products obtain a higher average ratings than low quality products regardless of rating biases or errors. However many empirical cases demonstrate that consumers may be misled by rating scores added by unreliable users or deliberate tampering. In this case, users' reputation, i.e., the ability to rating trustily and precisely, make a big difference during the evaluating process. Thus, one of the main challenges in designing reputation systems is eliminating the effects of users' rating bias on the evaluation results. To give an objective evaluation of each user's reputation and uncover an object's intrinsic quality, we propose an iterative balance (IB) method to correct users' rating biases. Experiments on two online video-provided Web sites, namely MovieLens and Netflix datasets, show that the IB method is a highly self-consistent and robust algorithm and it can accurately quantify movies' actual quality and users' stability of rating. Compared with existing methods, the IB method has higher ability to find the "dark horses", i.e., not so popular yet good movies, in the Academy Awards.