Good Question! The Effect of Positive Feedback on Contributions to Online Public Goods

TL;DR

This study shows that positive feedback via anonymous upvotes on Stack Overflow increases users' likelihood to ask and answer questions, with effects lasting up to twelve weeks, influenced by algorithmic visibility.

Contribution

The paper provides experimental evidence on how anonymous positive feedback impacts user engagement and participation in online public goods, highlighting the role of algorithmic amplification.

Findings

Upvotes increase the likelihood of asking questions by 6.3%.

Upvotes increase answering behavior by 12.9%.

Algorithmic amplification significantly affects answering but not asking.

Abstract

Online platforms where volunteers answer each other's questions are important sources of knowledge, yet participation is declining. We ran a pre-registered experiment on Stack Overflow, one of the largest Q&A communities for software development (N = 22,856), randomly assigning newly posted questions to receive an anonymous upvote. Within four weeks, treated users were 6.3% more likely to ask another question and 12.9% more likely to answer someone else's question. A second upvote produced no additional effect. The effect on answering was larger, more persistent, and still significant at twelve weeks. Next, we examine how much of these effects are due to algorithmic amplification, since upvotes also raise a question's rank and visibility. Algorithmic amplification is not important for the effect on asking additional questions, but it matters a lot for the effect on answering other questions. The increase in visibility increases the probability that another user provides an answer, and that experience appears to shift the poster toward broader community participation.