Reputation (In)dependence in Ranking Systems: Demographics Influence Over Output Disparities

TL;DR

This paper investigates how demographic attributes influence user reputation in ranking systems, revealing systematic disparities and proposing mitigation methods to ensure fairness and improve ranking accuracy.

Contribution

It introduces the concept of disparate reputation (DR), demonstrates its existence for gender and age, and proposes mitigation techniques to eliminate bias in reputation-based rankings.

Findings

DR systematically favors certain demographic groups.

Mitigation methods effectively reduce reputation disparities.

Fairness improvements also enhance ranking effectiveness.

Abstract

Recent literature on ranking systems (RS) has considered users' exposure when they are the object of the ranking. Although items are the object of reputation-based RS, users have a central role also in this class of algorithms. Indeed, when ranking the items, user preferences are weighted by how relevant this user is in the platform (i.e., their reputation). In this paper, we formulate the concept of disparate reputation (DR) and study if users characterized by sensitive attributes systematically get a lower reputation, leading to a final ranking that reflects less their preferences. We consider two demographic attributes, i.e., gender and age, and show that DR systematically occurs. Then, we propose mitigation, which ensures that reputation is independent of the users' sensitive attributes. Experiments on real-world data show that our approach can overcome DR and also improve ranking effectiveness.