Uncovering the Dynamics of Crowdlearning and the Value of Knowledge

TL;DR

This paper introduces a probabilistic model to analyze crowdlearning dynamics, revealing how user expertise evolves through contributions and assessments on platforms like Stack Overflow over several years.

Contribution

It presents a novel scalable probabilistic framework that models user expertise evolution, incorporating off-site learning, forgetting, and user assessments from large-scale data.

Findings

High-value answers are rare.

Middle-range users acquire more knowledge.

Prolific learners tend to contribute high-value answers.

Abstract

Learning from the crowd has become increasingly popular in the Web and social media. There is a wide variety of crowdlearning sites in which, on the one hand, users learn from the knowledge that other users contribute to the site, and, on the other hand, knowledge is reviewed and curated by the same users using assessment measures such as upvotes or likes. In this paper, we present a probabilistic modeling framework of crowdlearning, which uncovers the evolution of a user's expertise over time by leveraging other users' assessments of her contributions. The model allows for both off-site and on-site learning and captures forgetting of knowledge. We then develop a scalable estimation method to fit the model parameters from millions of recorded learning and contributing events. We show the effectiveness of our model by tracing activity of ~25 thousand users in Stack Overflow over a 4.5 year period. We find that answers with high knowledge value are rare. Newbies and experts tend to acquire less knowledge than users in the middle range. Prolific learners tend to be also proficient contributors that post answers with high knowledge value.