Detecting Sponsored Recommendations

TL;DR

This paper presents a statistical algorithm that enables a small group of users to detect hidden sponsored recommendations in online systems by analyzing binary feedback, addressing issues like search bias and disguised ads.

Contribution

It introduces a novel bias detection algorithm that works with minimal feedback and proves its effectiveness across various recommendation systems.

Findings

Algorithm detects bias with high probability

Effective with limited user feedback

Validated through extensive real data simulations

Abstract

With a vast number of items, web-pages, and news to choose from, online services and the customers both benefit tremendously from personalized recommender systems. Such systems however provide great opportunities for targeted advertisements, by displaying ads alongside genuine recommendations. We consider a biased recommendation system where such ads are displayed without any tags (disguised as genuine recommendations), rendering them indistinguishable to a single user. We ask whether it is possible for a small subset of collaborating users to detect such a bias. We propose an algorithm that can detect such a bias through statistical analysis on the collaborating users' feedback. The algorithm requires only binary information indicating whether a user was satisfied with each of the recommended item or not. This makes the algorithm widely appealing to real world issues such as identification of search engine bias and pharmaceutical lobbying. We prove that the proposed algorithm detects the bias with high probability for a broad class of recommendation systems when sufficient number of users provide feedback on sufficient number of recommendations. We provide extensive simulations with real data sets and practical recommender systems, which confirm the trade offs in the theoretical guarantees.