Rating Effects on Social News Posts and Comments

TL;DR

This study investigates how small, random rating manipulations influence social media content ratings, revealing significant herding effects that alter final ratings and the likelihood of reaching high ratings, with different impacts on posts and comments.

Contribution

It provides large-scale experimental evidence of how editorial ratings causally affect online human behavior and content evaluation in social media.

Findings

Positive treatments increase post ratings by 11.02%

Negative treatments decrease ratings by 5.15% for posts and 37.4% for comments

Positive ratings boost high-rating probability for posts by 24.6%

Abstract

At a time when information seekers first turn to digital sources for news and opinion, it is critical that we understand the role that social media plays in human behavior. This is especially true when information consumers also act as information producers and editors through their online activity. In order to better understand the effects that editorial ratings have on online human behavior, we report the results of a two large-scale in-vivo experiments in social media. We find that small, random rating manipulations on social media posts and comments created significant changes in downstream ratings resulting in significantly different final outcomes. We found positive herding effects for positive treatments on posts, increasing the final rating by 11.02% on average, but not for positive treatments on comments. Contrary to the results of related work, we found negative herding effects for negative treatments on posts and comments, decreasing the final ratings on average, of posts by 5.15% and of comments by 37.4%. Compared to the control group, the probability of reaching a high rating (>=2000) for posts is increased by 24.6% when posts receive the positive treatment and for comments is decreased by 46.6% when comments receive the negative treatment.