Enhancing LLM-Based Bug Reproduction for Android Apps via Pre-Assessment of Visual Effects

TL;DR

This paper introduces LTGDroid, a novel approach that enhances LLM-based bug reproduction in Android apps by pre-assessing visual effects, significantly improving success rates and efficiency.

Contribution

LTGDroid leverages visual effect recording during UI exploration to guide LLMs, addressing their limitations in interpreting UI visuals for bug reproduction.

Findings

Achieves 87.51% bug reproduction success rate

Improves success over baselines by 49.16% and 556.30%

Reproduces bugs in about 20 minutes at a cost of $0.27

Abstract

In the development and maintenance of Android apps, the quick and accurate reproduction of user-reported bugs is crucial to ensure application quality and improve user satisfaction. However, this process is often time-consuming and complex. Therefore, there is a need for an automated approach that can explore the Application Under Test (AUT) and identify the correct sequence of User Interface (UI) actions required to reproduce a bug, given only a complete bug report. Large Language Models (LLMs) have shown remarkable capabilities in understanding textual and visual semantics, making them a promising tool for planning UI actions. Nevertheless, our study shows that even when using state-of-the-art LLM-based approaches, these methods still struggle to follow detailed bug reproduction instructions and replan based on new information, due to their inability to accurately predict and interpret the visual effects of UI components. To address these limitations, we propose LTGDroid. Our insight is to execute all possible UI actions on the current UI page during exploration, record their corresponding visual effects, and leverage these visual cues to guide the LLM in selecting UI actions that are likely to reproduce the bug. We evaluated LTGDroid, instantiated with GPT-4.1, on a benchmark consisting of 75 bug reports from 45 popular Android apps. The results show that LTGDroid achieves a reproduction success rate of 87.51%, improving over the state-of-the-art baselines by 49.16% and 556.30%, while requiring an average of 20.45 minutes and approximately $0.27 to successfully reproduce a bug. The LTGDroid implementation is publicly available at https://github.com/N3onFlux/LTGDroid.