GBM Returns the Best Prediction Performance among Regression Approaches: A Case Study of Stack Overflow Code Quality

TL;DR

This study evaluates various regression techniques to predict Stack Overflow Java code quality, finding that Gradient Boosting Machine (GBM) offers the best predictive performance among tested approaches.

Contribution

It introduces a regression-based approach to predict Stack Overflow code quality and demonstrates that GBM outperforms other regression methods in this context.

Findings

GBM achieved the highest prediction accuracy among six regression approaches.

Longer code snippets tend to have more violations.

Questions with higher scores attract more views and answers, correlating with more code errors.

Abstract

Practitioners are increasingly dependent on publicly available resources for supporting their knowledge needs during software development. This has thus caused a spotlight to be paced on these resources, where researchers have reported mixed outcomes around the quality of these resources. Stack Overflow, in particular, has been studied extensively, with evidence showing that code resources on this platform can be of poor quality at times. Limited research has explored the variables or factors that predict code quality on Stack Overflow, but instead has focused on ranking content, identifying defects and predicting future content. In many instances approaches used for prediction are not evaluated to identify the best techniques. Contextualizing the Stack Overflow code quality problem as regression-based, we examined the variables that predict Stack Overflow (Java) code quality, and the regression approach that provides the best predictive power. Six approaches were considered in our evaluation, where Gradient Boosting Machine (GBM) stood out. In addition, longer Stack Overflow code tended to have more code violations, questions that were scored higher also attracted more views and the more answers that are added to questions on Stack Overflow the more errors were typically observed in the code that was provided. Outcomes here point to the value of the GBM ensemble learning mechanism, and the need for the practitioner community to be prudent when contributing and reusing Stack Overflow Java coding resource.