An Empirical Study of Obsolete Answers on Stack Overflow

TL;DR

This study analyzes how answers on Stack Overflow become obsolete over time, revealing that many are outdated when posted and rarely updated, especially in certain tags, highlighting the need for better answer maintenance.

Contribution

It provides an empirical analysis of answer obsolescence on Stack Overflow, identifying key characteristics and suggesting community-driven solutions.

Findings

58.4% of obsolete answers were already outdated when posted

Only 20.5% of obsolete answers are updated after becoming outdated

Answers in tags like node.js, ajax, android, are more prone to obsolescence

Abstract

Stack Overflow accumulates an enormous amount of software engineering knowledge. However, as time passes, certain knowledge in answers may become obsolete. Such obsolete answers, if not identified or documented clearly, may mislead answer seekers and cause unexpected problems (e.g., using an out-dated security protocol). In this paper, we investigate how the knowledge in answers becomes obsolete and identify the characteristics of such obsolete answers. We find that: 1) More than half of the obsolete answers (58.4%) were probably already obsolete when they were first posted. 2) When an obsolete answer is observed, only a small proportion (20.5%) of such answers are ever updated. 3) Answers to questions in certain tags (e.g., node.js, ajax, android, and objective-c) are more likely to become obsolete. Our findings suggest that Stack Overflow should develop mechanisms to encourage the whole community to maintain answers (to avoid obsolete answers) and answer seekers are encouraged to carefully go through all information (e.g., comments) in answer threads.