Enhancing Android Application Bug Reporting

TL;DR

This paper introduces Fusion, a system that improves Android bug reporting by auto-completing reproduction steps using static and dynamic analysis, resulting in more reproducible and actionable bug reports.

Contribution

Fusion is a novel system that leverages GUI analysis to assist reporters in creating detailed, reproducible bug reports for mobile apps.

Findings

Fusion significantly improves bug report reproducibility.

The system reduces reporting time for testers.

Enhanced bug reports lead to faster bug resolution.

Abstract

The modern software development landscape has seen a shift in focus toward mobile applications as smartphones and tablets near ubiquitous adoption. Due to this trend, the complexity of these "apps" has been increasing, making development and maintenance challenging. Current bug tracking systems do not effectively facilitate the creation of bug reports with useful information that will directly lead to a bug's resolution. To address the need for an improved reporting system, we introduce a novel solution, called Fusion, that helps reporters auto-complete reproduction steps in bug reports for mobile apps by taking advantage of their GUI-centric nature. Fusion links information, that reporters provide, to program artifacts extracted through static and dynamic analysis performed beforehand. This allows our system to facilitate the reporting process for developers and testers, while generating more reproducible bug reports with immediately actionable information.

Figures

Full text

††terms: Experimentation, Design

1 Motivation & Research Problem

Software maintenance activities are known to be generally expensive and challenging [20] and one of the most important maintenance tasks, especially in hyper-competitive mobile marketplaces, is bug report resolution. However, current bug tracking systems such as Bugzilla [1], Mantis [6], the Google Code Issue Tracker (GCIT) [4], the GitHub Issue Tracker [3], and commercial solutions such as JIRA [5] rely mostly on unstructured natural language bug descriptions.

The problem facing many of these current reporting systems is that typical natural language reports capture a coarse grained level of detail that makes developer reasoning about defects difficult. This highlights the underlying task that issue tracking systems must accomplish: bridging the lexical knowledge gap between typical reporters of a bug and the developers that must resolve the bugs. Previous studies on bug report quality and developer information needs highlight several factors that can impact the quality of reports [10, 13, 9]: 1) Other than “Interbug dependencies” (i.e., a situation where a bug was fixed in a previous patch), insufficient information in bug reports is one of the leading causes of non-reproducible bug reports [13]; 2) Developers consider (i)steps to reproduce, (ii)stack traces, and (iii)test cases/scenarios as the most helpful sources of information in a bug report [9]; 3) Information needs are greatest early in a bug’s life cycle, therefore, a way to easily add the above features is important during bug report creation [10].

Using these issues as motivation, we developed an augmented bug reporting solution, Fusion, that gleans contextual information from the GUI of a mobile app to accomplish the following goals: (i) provide bug reports to developers with immediately actionable knowledge (reliable reproduction steps) and (ii) facilitate reporting by providing this information through an auto-completion mechanism.

2 Background & Related Work

In-Field Failure Reproduction: A body of work known as in-field failure reproduction [8, 15, 21, 12, 14, 7, 16, 11] shares similar goals with our approach. These techniques collect run-time information (e.g., execution traces) from instrumented programs that provide developers with a better understanding of the causes of a failure, which subsequently helps expedite the fixing of the corresponding faults. However, there are three key points that set our work apart and illustrate how Fusion improves on the state of research.

First, in-field failure reproduction techniques rely on potentially expensive program instrumentation. Fusion is completely automatic, our static and dynamic analysis techniques only need to be applied once for the version of the program that is released for testing. Second, current in-field failure reproduction techniques require an oracle to signify when a failure has occurred (e.g., a crash). Fusion is not an approach for crash or failure detection, it is designed to support testers during the bug reporting process. Third, these techniques have not been applied to mobile apps and would likely need to be optimized further to be applicable for the corresponding resource-constrained environment.

3 Approach

In this section we present our approach for Android bug reporting that utilizes an Analyze $\rightarrow$ Generate workflow corresponding to two major phases. In the Analysis Phase Fusion collects app and GUI information and in the Report Generation Phase, Fusion operates on this information to aid users in constructing bug reports. The overall design of Fusion can be seen in Figure 1. Please refer to the following materials [18, 19] for a complete description and dataset.

3.1 Analysis Phase

The Analysis Phase has two major components: 1) static analysis (Primer), and 2) dynamic program analysis (Engine) of a target app. Both components store their data in the Fusion database (Fig. 1 -
3 ).

Static Analysis (Primer) The goal of the Primer (Fig. 1 -
1 ) is to extract all of the GUI components and associated information from the app source code. For each GUI component, the Primer extracts: (i) possible actions on that component, (ii) type of the component (e.g., Button, Spinner), (iii) activities the component is contained within, and (iv) class files where the component is instantiated. Thus, this phase results in a universe of possible GUI components for the application, and establishes traceability links connecting these components to specific code-level artifacts.

Dynamic Analysis (Engine) The Engine (Fig. 1 -
2 ) is used to glean dynamic contextual information, such as the location of the GUI component on the screen, and enhance the database with both run-time GUI and application event flow information. The goal of the Engine is to explore an app in a systematic manner ripping and extracting run-time information related to the GUI components during execution including: (i) text associated with a component, (ii) contextual screenshots of the component, (iii) the activity or screen the component is located within, and (iv) the location of the component on a given app screen. During the ripping, before each step is executed on the GUI, the Engine calls UIAutomator subroutines to extract the contextual information outlined above regarding each GUI component on the screen. To effectively explore and model the application we took inspiration from our previous work [17] and implemented a systematic depth-first search (DFS) algorithm that performs click/tap events on the components in the GUI hierarchy for a given app.

3.2 Report Generation Phase

The design of the Report Generation Phase component of Fusion has two goals: 1) Allow for traditional natural language input in order to give a high-level overview of a bug. 2) Auto-complete the reproduction steps of a bug through suggestions derived by tracking the position of the reporter’s step entry in the app event flow. During the Report Generation Phase Fusion’s Auto-completion engine populates a decision tree based on the extracted app model and aids the reporter in constructing the steps needed to recreate a bug by making suggestions based upon the “potential" GUI state reached by the declared steps. This means for each step $s$, Fusion infers – online – the GUI state $GUI_{s}$ in which the target app should be by taking into account the step history. For each step, Fusion verifies that the suggestion made to the reporter is correct by asking the reporter to select a full contextual screenshot corresponding to the {action ,gui-component} tuple they intended to record. The end result is a detailed bug report including natural-language general information and contextualized reproduction steps with screen-shots for each step and traceability links from steps to the app source code.

4 Results and Contributions

We have evaluated the proposed framework with two empirical studies using 15 real bugs in 14 open-source Android applications extracted from the F-Droid marketplace [2] comparing Fusion against the GCIT [4] and the original bug reports in two maintenance activities involving reporting (Study 1) and reproducing bugs (Study 2). Study 1 involved eight graduate students from the College of William & Mary (W&M) who created reports from videos of the bugs for the two systems, for a total of 120 (60 Fusion, 60 GCIT) bug reports. In Study 2, 20 new participants, also W&M students, attempted to reproduce the 15 bugs from the bug reports created in the first study, along with the original issue tracker reports. Our results show that, overall Fusion produces more reproducible bug reports than current issue trackers (13/120 non-reproducible with Fusion, versus 23/120 non-reproducible with GCIT) . The major contribution of this work is the following conclusion: By leveraging program analysis and the GUI-centric nature of mobile apps, mobile issue trackers can be augmented to facilitate users creating more detailed and reproducible bug reports.