A new perspective on the competent programmer hypothesis through the reproduction of bugs with repeated mutations

TL;DR

This paper investigates the competent programmer hypothesis by attempting to reproduce real-world bugs through mutation chains, revealing that mutation testing may lack operators needed to generate representative bugs.

Contribution

It offers a new perspective on the hypothesis by analyzing mutation paths and highlights gaps in mutation testing operators for bug reproduction.

Findings

Mutation chains can approximate real-world bugs

Mutation testing may miss key operators for bug generation

The competent programmer hypothesis is generally supported

Abstract

The competent programmer hypothesis states that most programmers are competent enough to create correct or almost correct source code. Because this implies that bugs should usually manifest through small variations of the correct code, the competent programmer hypothesis is one of the fundamental assumptions of mutation testing. Unfortunately, it is still unclear if the competent programmer hypothesis holds and past research presents contradictory claims. Within this article, we provide a new perspective on the competent programmer hypothesis and its relation to mutation testing. We try to re-create real-world bugs through chains of mutations to understand if there is a direct link between mutation testing and bugs. The lengths of these paths help us to understand if the source code is really almost correct, or if large variations are required. Our results indicate that while the competent programmer hypothesis seems to be true, mutation testing is missing important operators to generate representative real-world bugs.