Automated Generation of High-Quality Bug Reports for Android Applications

TL;DR

This paper introduces BugScribe, an automated system that leverages LLMs and app-specific context to generate high-quality, detailed bug reports for Android applications, improving clarity and accuracy over manual reports.

Contribution

The paper presents BugScribe, a novel approach that uses app context with LLMs to automatically produce comprehensive bug reports, outperforming existing methods.

Findings

BugScribe generates more accurate bug report components than original reports.

It outperforms recent LLM-based baselines in quality and correctness.

The approach enhances bug report clarity, aiding faster bug resolution.

Abstract

Most defects in mobile applications are visually observable on the device screen. To track these defects, users, testers, and developers must manually submit bug reports, especially in the absence of crashes. However, these reports are frequently ambiguous or inaccurate, often omitting essential components such as the Observed Behavior (OB), Expected Behavior (EB), or Steps to Reproduce (S2Rs). Low-quality reports hinder developers' ability to understand and reproduce defects, delaying resolution and leading to incorrect or unresolvable fixes. In this paper, we posit that providing specific app-related information (e.g., GUI interactions or specific screens where bugs appear) to LLMs as key points of context can assist in automatically generating clear, detailed, and accurate OB, EB, and S2Rs. We built and evaluated a novel approach, BugScribe, that generates bug reports in this way. To support the evaluation, we introduce a unified quality framework that defines correctness and completeness dimensions for OB, EB, and S2Rs. Using 48 bug reports from 26 Android apps, we show that BugScribe produces higher-quality and more accurate components than the original reports and outperforms recent LLM-based baselines. We envision that BugScribe can serve as a practical assistant for testers and developers by enhancing incomplete bug reports with reliable and accurate OB, EB, and S2Rs, thereby streamlining bug resolution and improving mobile app quality.