SOTorrent: Reconstructing and Analyzing the Evolution of Stack Overflow Posts

TL;DR

SOTorrent is an open dataset that captures the detailed version history of Stack Overflow posts, enabling analysis of how questions and answers evolve over time, including code and text edits, and their connections to other platforms.

Contribution

This paper introduces SOTorrent, a novel dataset that reconstructs Stack Overflow post histories at fine granularity and evaluates string similarity metrics for accurate version reconstruction.

Findings

Post edits are typically small and occur shortly after creation.

Code snippets are rarely changed without editing surrounding text.

There is a strong correlation between post edits and comments.

Abstract

Stack Overflow (SO) is the most popular question-and-answer website for software developers, providing a large amount of code snippets and free-form text on a wide variety of topics. Like other software artifacts, questions and answers on SO evolve over time, for example when bugs in code snippets are fixed, code is updated to work with a more recent library version, or text surrounding a code snippet is edited for clarity. To be able to analyze how content on SO evolves, we built SOTorrent, an open dataset based on the official SO data dump. SOTorrent provides access to the version history of SO content at the level of whole posts and individual text or code blocks. It connects SO posts to other platforms by aggregating URLs from text blocks and by collecting references from GitHub files to SO posts. In this paper, we describe how we built SOTorrent, and in particular how we evaluated 134 different string similarity metrics regarding their applicability for reconstructing the version history of text and code blocks. Based on a first analysis using the dataset, we present insights into the evolution of SO posts, e.g., that post edits are usually small, happen soon after the initial creation of the post, and that code is rarely changed without also updating the surrounding text. Further, our analysis revealed a close relationship between post edits and comments. Our vision is that researchers will use SOTorrent to investigate and understand the evolution of SO posts and their relation to other platforms such as GitHub.