On Debugging the Performance of Configurable Software Systems: Developer Needs and Tailored Tool Support

TL;DR

This paper explores developers' needs when debugging performance issues in configurable software systems, and presents a tailored tool supported by empirical studies that improves debugging effectiveness.

Contribution

It identifies developers' information needs and introduces a tailored debugging tool, validated through user studies, addressing a gap in usability-focused evaluation.

Findings

Developers have specific information needs during performance debugging.

The tailored tool effectively supports developers in debugging tasks.

Empirical evidence shows improved debugging performance with the tool.

Abstract

Determining whether a configurable software system has a performance bug or it was misconfigured is often challenging. While there are numerous debugging techniques that can support developers in this task, there is limited empirical evidence of how useful the techniques are to address the actual needs that developers have when debugging the performance of configurable software systems; most techniques are often evaluated in terms of technical accuracy instead of their usability. In this paper, we take a human-centered approach to identify, design, implement, and evaluate a solution to support developers in the process of debugging the performance of configurable software systems. We first conduct an exploratory study with 19 developers to identify the information needs that developers have during this process. Subsequently, we design and implement a tailored tool, adapting techniques from prior work, to support those needs. Two user studies, with a total of 20 developers, validate and confirm that the information that we provide helps developers debug the performance of configurable software systems.