Can Who-Edits-What Predict Edit Survival?

TL;DR

This paper introduces a simple, general model to predict whether an edit in peer-production systems like Wikipedia and Linux kernel will survive, based on who made the edit and the nature of the component affected.

Contribution

The paper proposes a novel, content-independent model for predicting edit survival that outperforms reputation-based methods and approaches specialized content-based predictors.

Findings

Model accurately predicts edit survival in Wikipedia and Linux kernel.

Outperforms reputation-based prediction methods.

Bridges the gap with content-based predictors.

Abstract

As the number of contributors to online peer-production systems grows, it becomes increasingly important to predict whether the edits that users make will eventually be beneficial to the project. Existing solutions either rely on a user reputation system or consist of a highly specialized predictor that is tailored to a specific peer-production system. In this work, we explore a different point in the solution space that goes beyond user reputation but does not involve any content-based feature of the edits. We view each edit as a game between the editor and the component of the project. We posit that the probability that an edit is accepted is a function of the editor's skill, of the difficulty of editing the component and of a user-component interaction term. Our model is broadly applicable, as it only requires observing data about who makes an edit, what the edit affects and whether the edit survives or not. We apply our model on Wikipedia and the Linux kernel, two examples of large-scale peer-production systems, and we seek to understand whether it can effectively predict edit survival: in both cases, we provide a positive answer. Our approach significantly outperforms those based solely on user reputation and bridges the gap with specialized predictors that use content-based features. It is simple to implement, computationally inexpensive, and in addition it enables us to discover interesting structure in the data.