Learning Actionable Analytics from Multiple Software Projects

TL;DR

This paper introduces XTREE and BELLTREE, two planning tools that generate actionable recommendations to reduce software defects, demonstrating effectiveness across multiple open-source Java projects.

Contribution

The paper presents novel planning methods, XTREE and BELLTREE, that provide actionable guidance for defect reduction within and across software projects, outperforming existing planners.

Findings

XTREE's plans are shorter and easier to implement.

Plans from XTREE significantly reduce defects in open-source Java projects.

XTREE outperforms other planning methods in effectiveness.

Abstract

The current generation of software analytics tools are mostly prediction algorithms (e.g. support vector machines, naive bayes, logistic regression, etc). While prediction is useful, after prediction comes planning about what actions to take in order to improve quality. This research seeks methods that generate demonstrably useful guidance on "what to do" within the context of a specific software project. Specifically, we propose XTREE (for within-project planning) and BELLTREE (for cross-project planning) to generating plans that can improve software quality. Each such plan has the property that, if followed, it reduces the expected number of future defect reports. To find this expected number, planning was first applied to data from release x. Next, we looked for changes in release x+1 that conformed to our plans. This procedure was applied using a range of planners from the literature, as well as XTREE. In 10 open-source JAVA systems, several hundreds of defects were reduced in sections of the code that conformed to XTREE's plans. Further, when compared to other planners, XTREE's plans were found to be easier to implement (since they were shorter) and more effective at reducing the expected number of defects.