A Bayesian Framework for Automated Debugging

TL;DR

This paper introduces a Bayesian framework for understanding automated debugging, enabling formal analysis of techniques like fault localization and program repair, and demonstrates its effectiveness through a new patch prioritization method that improves accuracy and efficiency.

Contribution

It presents the first theoretical Bayesian framework for automated debugging, unifying and analyzing existing techniques, and proposes BAPP, a value-aware patch prioritization method that enhances debugging performance.

Findings

BAPP reduces patch validation by 68%.

BAPP decreases repair time by 34 minutes on average.

BAPP increases fault localization accuracy from 8 to 11 at@5.

Abstract

Debugging takes up a significant portion of developer time. As a result, automated debugging techniques including Fault Localization (FL) and Automated Program Repair (APR) have garnered significant attention due to their potential to aid developers in debugging tasks. Despite intensive research on these subjects, we are unaware of a theoretic framework that highlights the principles behind automated debugging and allows abstract analysis of techniques. Such a framework would heighten our understanding of the endeavor and provide a way to formally analyze techniques and approaches. To this end, we first propose a Bayesian framework of understanding automated repair and find that in conjunction with a concrete statement of the objective of automated debugging, we can recover maximal fault localization formulae from prior work, as well as analyze existing APR techniques and their underlying assumptions. As a means of empirically demonstrating our framework, we further propose BAPP, a Bayesian Patch Prioritization technique that incorporates intermediate program values to analyze likely patch locations and repair actions, with its core equations being derived by our Bayesian framework. We find that incorporating program values allows BAPP to identify correct patches more precisely: when applied to the patches generated by kPAR, the rankings produced by BAPP reduce the number of required patch validation by 68% and consequently reduce the repair time by 34 minutes on average. Further, BAPP improves the precision of FL, increasing acc@5 on the studied bugs from 8 to 11. These results highlight the potential of value-cognizant automated debugging techniques, and further validates our theoretical framework. Finally, future directions that the framework suggests are provided.