Using Hypotheses as a Debugging Aid

TL;DR

This paper investigates how hypotheses influence debugging success, revealing that providing potential hypotheses significantly improves developers' debugging effectiveness, while fault location hints do not.

Contribution

It introduces empirical evidence on the role of hypotheses in debugging and shows that sharing potential hypotheses can greatly enhance debugging success.

Findings

Providing potential hypotheses increases debugging success sixfold.

Developers typically hold two hypotheses per defect.

Early correct hypotheses predict later debugging success.

Abstract

As developers debug, developers formulate hypotheses about the cause of the defect and gather evidence to test these hypotheses. To better understand the role of hypotheses in debugging, we conducted two studies. In a preliminary study, we found that, even with the benefit of modern internet resources, incorrect hypotheses can cause developers to investigate irrelevant information and block progress. We then conducted a controlled experiment where 20 developers debugged and recorded their hypotheses. We found that developers have few hypotheses, two per defect. Having a correct hypothesis early strongly predicted later success. We also studied the impact of two debugging aids: fault locations and potential hypotheses. Offering fault locations did not help developers formulate more correct hypotheses or debug more successfully. In contrast, offering potential hypotheses made developers six times more likely to succeed. These results demonstrate the potential of future debugging tools that enable finding and sharing relevant hypotheses.