On the Rise and Fall of Simple Stupid Bugs: a Life-Cycle Analysis of SStuBs

TL;DR

This paper analyzes the life-cycle of simple, easily fixable bugs in Java codebases, revealing their long presence, developer fix patterns, and limitations of current detection tools, to improve software quality assurance.

Contribution

It provides an empirical study of SStuBs, highlighting their persistence, fix patterns, and the inadequacy of existing bug detection tools like PMD and SpotBugs.

Findings

Most SStuBs remain unnoticed for a long time.

Less than half of SStuBs are fixed by the original developer.

Current tools like PMD and SpotBugs are insufficient for detecting SStuBs.

Abstract

Bug detection and prevention is one of the most important goals of software quality assurance. Nowadays, many of the major problems faced by developers can be detected or even fixed fully or partially with automatic tools. However, recent works explored that there exists a substantial amount of simple yet very annoying errors in code-bases, which are easy to fix, but hard to detect as they do not hinder the functionality of the given product in a major way. Programmers introduce such errors accidentally, mostly due to inattention. Using the ManySStuBs4J dataset, which contains many simple, stupid bugs, found in GitHub repositories written in the Java programming language, we investigated the history of such bugs. We were interested in properties such as: How long do such bugs stay unnoticed in code-bases? Whether they are typically fixed by the same developer who introduced them? Are they introduced with the addition of new code or caused more by careless modification of existing code? We found that most of such stupid bugs lurk in the code for a long time before they get removed. We noticed that the developer who made the mistake seems to find a solution faster, however less then half of SStuBs are fixed by the same person. We also examined PMD's performance when to came to flagging lines containing SStuBs, and found that similarly to SpotBugs, it is insufficient when it comes to finding these types of errors. Examining the life-cycle of such bugs allows us to better understand their nature and adjust our development processes and quality assurance methods to better support avoiding them.