Features that Predict the Acceptability of Java and JavaScript Answers on Stack Overflow

TL;DR

This study identifies key features such as code length, user reputation, text similarity, and response time that predict whether an answer on Stack Overflow will be accepted, aiding in better answer selection.

Contribution

It introduces a predictive model using machine learning to distinguish accepted answers from unaccepted ones based on answer characteristics.

Findings

Code length and user reputation are strong predictors.

Text similarity between question and answer influences acceptance.

Time lag between question posting and answer impacts acceptance likelihood.

Abstract

Context: Stack Overflow is a popular community question and answer portal used by practitioners to solve problems during software development. Developers can focus their attention on answers that have been accepted or where members have recorded high votes in judging good answers when searching for help. However, the latter mechanism (votes) can be unreliable, and there is currently no way to differentiate between an answer that is likely to be accepted and those that will not be accepted by looking at the answer's characteristics. Objective: In potentially providing a mechanism to identify acceptable answers, this study examines the features that distinguish an accepted answer from an unaccepted answer. Methods: We studied the Stack Overflow dataset by analyzing questions and answers for the two most popular tags (Java and JavaScript). Our dataset comprised 249,588 posts drawn from 2014-2016. We use random forest and neural network models to predict accepted answers, and study the features with the highest predictive power in those two models. Results: Our findings reveal that the length of code in answers, reputation of users, similarity of the text between questions and answers, and the time lag between questions and answers have the highest predictive power for differentiating accepted and unaccepted answers. Conclusion: Tools may leverage these findings in supporting developers and reducing the effort they must dedicate to searching for suitable answers on Stack Overflow.