Auditing Yelp's Business Ranking and Review Recommendation Through the Lens of Fairness
Mohit Singhal, Javier Pacheco, Seyyed Mohammad Sadegh Moosavi, Khorzooghi, Tanushree Debi, Abolfazl Asudeh, Gautam Das, Shirin Nilizadeh

TL;DR
This paper conducts a data-driven audit of Yelp's business ranking and review recommendation systems to identify potential biases against less-established users and certain neighborhoods, revealing significant disparities.
Contribution
First comprehensive analysis of Yelp's algorithms for bias, highlighting disparities based on user experience and neighborhood demographics.
Findings
Reviews of less-established users are often not recommended.
Restaurants in hotspot regions receive higher exposure.
Biases are more severe in affluent, less diverse neighborhoods.
Abstract
Auditing is critical to ensuring the fairness and reliability of decision-making systems. However, auditing a black-box system for bias can be challenging due to the lack of transparency in the model's internal workings. In many web applications, such as Yelp, it is challenging, if not impossible, to manipulate their inputs systematically to identify bias in the output. Yelp connects users and businesses, where users identify new businesses and simultaneously express their experiences through reviews. Yelp recommendation software moderates user-provided content by categorizing it into recommended and not-recommended sections. The recommended reviews, among other attributes, are used by Yelp's ranking algorithm to rank businesses in a neighborhood. Due to Yelp's substantial popularity and its high impact on local businesses' success, understanding the bias of its algorithms is crucial.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDigital Marketing and Social Media · Technology Adoption and User Behaviour · Customer Service Quality and Loyalty
