An Empirical Study on Leveraging Images in Automated Bug Report Reproduction

TL;DR

This empirical study investigates the role of images in bug reports, revealing their significance and functional roles in improving automated bug reproduction, and analyzing how they impact existing tools.

Contribution

The paper provides a comprehensive analysis of images in bug reports, identifying their types, patterns, and functional roles, and evaluates their influence on bug reproduction tools.

Findings

Six functional roles of images identified

Images significantly improve bug reproduction success

Patterns of images correlate with bug types

Abstract

Automated bug reproduction is a challenging task, with existing tools typically relying on textual steps-to-reproduce, videos, or crash logs in bug reports as input. However, images provided in bug reports have been overlooked. To address this gap, this paper presents an empirical study investigating the necessity of including images as part of the input in automated bug reproduction. We examined the characteristics and patterns of images in bug reports, focusing on (1) the distribution and types of images (e.g., UI screenshots), (2) documentation patterns associated with images (e.g., accompanying text, annotations), and (3) the functional roles they served, particularly their contribution to reproducing bugs. Furthermore, we analyzed the impact of images on the performance of existing tools, identifying the reasons behind their influence and the ways in which they can be leveraged to improve bug reproduction. Our findings reveal several key insights that demonstrate the importance of images in supporting automated bug reproduction. Specifically, we identified six distinct functional roles that images serve in bug reports, each exhibiting unique patterns and specific contributions to the bug reproduction process. This study offers new insights into tool advancement and suggests promising directions for future research.