Analyzing Item Popularity Bias of Music Recommender Systems: Are Different Genders Equally Affected?

TL;DR

This study investigates how music recommender systems exhibit popularity bias and whether this bias affects different genders equally, using detailed statistical measures and real user data from Last.fm.

Contribution

It introduces comprehensive statistical analysis of popularity bias and examines gender-based differences in bias amplification in music recommendation algorithms.

Findings

Metrics reveal nuanced insights into popularity bias beyond mean differences.

Less biased algorithms do not necessarily have lower utility.

Female users are more affected by popularity bias amplification.

Abstract

Several studies have identified discrepancies between the popularity of items in user profiles and the corresponding recommendation lists. Such behavior, which concerns a variety of recommendation algorithms, is referred to as popularity bias. Existing work predominantly adopts simple statistical measures, such as the difference of mean or median popularity, to quantify popularity bias. Moreover, it does so irrespective of user characteristics other than the inclination to popular content. In this work, in contrast, we propose to investigate popularity differences (between the user profile and recommendation list) in terms of median, a variety of statistical moments, as well as similarity measures that consider the entire popularity distributions (Kullback-Leibler divergence and Kendall's tau rank-order correlation). This results in a more detailed picture of the characteristics of popularity bias. Furthermore, we investigate whether such algorithmic popularity bias affects users of different genders in the same way. We focus on music recommendation and conduct experiments on the recently released standardized LFM-2b dataset, containing listening profiles of Last.fm users. We investigate the algorithmic popularity bias of seven common recommendation algorithms (five collaborative filtering and two baselines). Our experiments show that (1) the studied metrics provide novel insights into popularity bias in comparison with only using average differences, (2) algorithms less inclined towards popularity bias amplification do not necessarily perform worse in terms of utility (NDCG), (3) the majority of the investigated recommenders intensify the popularity bias of the female users.