Disentangling the Effects of Social Signals

TL;DR

This paper experimentally investigates how social signals influence peer recommendation outcomes, revealing their significant role in popularity dynamics, preference bias, and efficiency improvements in content evaluation.

Contribution

It provides empirical evidence on the effects of social signals in peer recommendation, quantifying their impact relative to content and position, and analyzing their influence on popularity and decision-making.

Findings

Social signals account for about half the effect of position and content on popularity.

Social signals amplify the 'rich get richer' phenomenon, increasing popularity variance.

Social signals enhance recommendation efficiency while maintaining quality.

Abstract

Peer recommendation is a crowdsourcing task that leverages the opinions of many to identify interesting content online, such as news, images, or videos. Peer recommendation applications often use social signals, e.g., the number of prior recommendations, to guide people to the more interesting content. How people react to social signals, in combination with content quality and its presentation order, determines the outcomes of peer recommendation, i.e., item popularity. Using Amazon Mechanical Turk, we experimentally measure the effects of social signals in peer recommendation. Specifically, after controlling for variation due to item content and its position, we find that social signals affect item popularity about half as much as position and content do. These effects are somewhat correlated, so social signals exacerbate the "rich get richer" phenomenon, which results in a wider variance of popularity. Further, social signals change individual preferences, creating a "herding" effect that biases people's judgments about the content. Despite this, we find that social signals improve the efficiency of peer recommendation by reducing the effort devoted to evaluating content while maintaining recommendation quality.