The Bandwagon Effect: Not Just Another Bias

TL;DR

This paper challenges the common view of the bandwagon effect as a bias in recommender systems, showing it does not bias individual interactions but can cause inconsistency in estimators, highlighting an underexplored problem.

Contribution

It provides a theoretical analysis demonstrating that the bandwagon effect does not bias interactions but can lead to estimator inconsistency, distinguishing it from selection bias.

Findings

Bandwagon effect leaves individual interactions unbiased.

It can cause inconsistency in relevance estimators.

Theoretical conditions under which the effect impacts convergence.

Abstract

Optimizing recommender systems based on user interaction data is mainly seen as a problem of dealing with selection bias, where most existing work assumes that interactions from different users are independent. However, it has been shown that in reality user feedback is often influenced by earlier interactions of other users, e.g. via average ratings, number of views or sales per item, etc. This phenomenon is known as the bandwagon effect. In contrast with previous literature, we argue that the bandwagon effect should not be seen as a problem of statistical bias. In fact, we prove that this effect leaves both individual interactions and their sample mean unbiased. Nevertheless, we show that it can make estimators inconsistent, introducing a distinct set of problems for convergence in relevance estimation. Our theoretical analysis investigates the conditions under which the bandwagon effect poses a consistency problem and explores several approaches for mitigating these issues. This work aims to show that the bandwagon effect poses an underinvestigated open problem that is fundamentally distinct from the well-studied selection bias in recommendation.