Detecting and Diagnosing Energy Issues for Mobile Applications

TL;DR

This paper presents a novel testing framework that detects energy issues in mobile apps by analyzing workloads and contexts, revealing many previously unknown problems that significantly impact battery life.

Contribution

The paper introduces a new testing framework combining machine learning and context-aware testing to identify energy issues overlooked by existing methods.

Findings

91.6% of detected issues were previously unknown to developers

Energy issues can double app energy consumption

The framework achieves low false positive rates

Abstract

Energy efficiency is an important criterion to judge the quality of mobile apps, but one third of our randomly sampled apps suffer from energy issues that can quickly drain battery power. To understand these issues, we conducted an empirical study on 27 well-maintained apps such as Chrome and Firefox, whose issue tracking systems are publicly accessible. Our study revealed that the main root causes of energy issues include unnecessary workload and excessively frequent operations. Surprisingly, these issues are beyond the application of present technology on energy issue detection. We also found that 20.6% of energy issues can only manifest themselves under specific contexts such as poor network performance, but such contexts are again neglected by present technology. Therefore, we proposed a novel testing framework for detecting energy issues in real-world apps. Our framework examines apps with well-designed input sequences and runtime contexts. To identify the root causes mentioned above, we employed a machine learning algorithm to cluster the workloads and further evaluate their necessity. For the issues concealed by the specific contexts, we carefully set up several execution contexts to pinpoint them. More importantly, we developed leading edge technology, e.g. pre-designing input sequences with potential energy overuse and tuning tests on-the-fly, to achieve high efficacy in detecting energy issues. A large-scale evaluation shows that 91.6% issues detected in our test were previously unknown to developers. On average, these issues double the energy costs of the apps. Furthermore, our test achieves a low number of false positives. Finally, we show how our test reports can help developers fix the issues.

Figures

Tables

Table 1. Table 1: Comparison with present technology.

Root Cause	Present	This work
Unnecessary Workload    (42.3%)	✗	✓
Excessively Frequent Operations (19.7%)	✗	✓
Wasted Background Processing   (18.3%)	✓	✓
No Sleep             (25.4%)	✓	✓
Manifestation Type	Present	This work
Simple Inputs        (6.3%)	✓	✓
Special Inputs      (73.0%)	✓	✓
Special Context      (20.6%)	✗	✓

Table 2. Table 2: Examples of detected energy issues.

			Activity or	Hit	Context or	Energy	Reported
Application	Category	Issue Type	Symptom	Rate1	NonBg2	Waste	before?
Leisure	News	Execution	3 .gif playing on page	1.0%	Normal	25.9%	No
GPS Status	Travel	Execution	Not obvious	47.0%	Normal	15.3%	No
BatteryDog	Tools	Execution	Editing lengthy text	1.0%	Normal	30.8%	No
Rocket Chat	Comm.	Execution	Connect to server	1.0%	Normal	12.8%	No
Chrome	Browser	Execution	Keep loading pages	4.0%	WiFi Fail	9.7%	No
Chess Clock	Tools	Background	Device heated up	100.0%	WiFi Fail	59.2%	No
Vanilla	Multimedia	No Sleep	Enqueue many tracks	59.0%	WiFi Fail	242.8%	No
cgeo	Travel	No Sleep	App get stuck	2.0%	Flight Mode	179.4%	Yes
AntennaPod	Multimedia	No Sleep	Popping up messages	1.0%	NonBg	184.4%	Yes
1. Hit rate here is the percentage of test cases detected having the energy issue in that app. 2. “NonBg" is “Non-background".

Equations

$$weight=\alpha \ast S+\beta \ast C$$

$$w=(\frac{e_x}{e_n}-1)\times 100\%$$

Full text

Therefore, we proposed a novel testing framework for detecting energy issues in real-world apps. Our framework examines apps with well-designed input sequences and runtime contexts. To identify the root causes mentioned above, we employed a machine learning algorithm to cluster the workloads and further evaluate their necessity. For the issues concealed by the specific contexts, we carefully set up several execution contexts to pinpoint them. More importantly, we developed leading edge technology, e.g. pre-designing input sequences with potential energy overuse and tuning tests on-the-fly, to achieve high efficacy in detecting energy issues. A large-scale evaluation shows that 91.6% issues detected in our test were previously unknown to developers. On average, these issues double the energy costs of the apps. Furthermore, our test achieves a low number of false positives. Finally, we show how our test reports can help developers fix the issues.

1 Introduction

Recent years have seen a huge expansion of mobile devices such as smartphones and tablets. Due to the fact that mobile devices have limited battery capacity, energy efficiency has become one of the main considerations in system and app design. However, many mobile apps on the market are not well optimized and can waste much battery power. We randomly sampled 89 open-source Android apps and found that 27 (30.3%) of them suffer from serious software energy issues. Our experimental results (in Section 6) show that there still exist a large number of hidden energy issues in the apps which were unknown to developers and which cause significant battery drain.

In this paper, we aim to design an effective issue-detection technology to uncover these serious issues. Essentially, knowing what are the root causes and manifestation of energy issues determines the design and implementation of the technology. For example, if a large number of energy issues are caused by runtime exceptions, then the issue-detection technology can be conveniently integrated to the current exception-handling framework. This motivates us to conduct an empirical study on the 27 energy-inefficient Android apps to learn the real root causes of energy issues and their manifestation in practice. Specifically, we aim to answer two research questions:

• •

**RQ1 (Issue Causes): **What are the common root causes of energy issues?

• •

**RQ2 (Issue Manifestation): ***How do energy issues manifest themselves in practice? *

We studied 134 energy issues from the 27 open-source projects. Two findings inspired us to design a novel testing framework for detecting energy issues:

• •

Finding 1 (w.r.t. RQ1): we identified the main root causes of energy issues as unnecessary workload, excessively frequent operations, wasted background processing and no-sleep.

• •

Finding 2 (w.r.t. RQ2): we identified the main types of issue manifestation as simple inputs, special inputs and special contexts.

Table 1 compares present technology [2] and our work in terms of ability to deal with root causes and manifestations (and their proportions in the real-world). [2] is based on an assumption that the main cause of energy issues is high “E/U" ratio (the ratio of energy-consumption to hardware-utilization). If the ratio is high, it means energy consumption is high, while hardware utilization is low, implying that the app is energy-inefficient and suffering from energy issues. This point of view seems reasonable; the resulting technology is able to detect issues caused by wasted background processing and no-sleep, where E/U ratio is remarkably high. Our large-scale empirical study shows that the main root causes also include unnecessary workload and excessively frequent operations. These issues are beyond the application of E/U ratio since the ratio can not indicate the necessity of app workload. Thus, these issues usually enlarge E and U simultaneously, so they do not increase the ratio. Hence, their technology loses efficacy in detecting such energy issues.

Regarding Finding 2, present technology is capable of handling simple and special inputs. However, researchers have never considered special contexts (such as poor network performance) that hide as many as 20.6% of energy issues. Thus the energy-saving potential of considering such factors has not been researched sufficiently.

Based on these observations, we propose a novel testing framework for effectively detecting energy issues. Generally, our framework examines apps with a large variety of well-designed input sequences and runtime contexts, which typically provoke energy issues to manifest themselves, as shown in Finding 2. To identify issues caused by unnecessary workload and excessively frequent operations, we employ a machine learning algorithm to classify the workloads, and further assess their necessity according to several key criteria, such as lengths of continuous-high-power periods. If the workload is assessed unnecessary and severely energy consuming, then it will be identified as an energy issue. On the other hand, with respect to issues concealed by special contexts, we carefully devise several targeting runtime contexts to reveal them.

To enhance the issue-detection efficacy of our framework, we also designed and implemented features such as scanning and analysing the source code of apps to extract the input sequences that are most likely to provoke energy issues. The framework then explores the extracted input sequences with carefully-designed runtime contexts, which are dynamically adjusted to increase the chances of encountering energy issues.

The experimental results show that our tests can uncover a large number of serious energy issues in high-quality test apps, 91.6% of which have never been discovered before. On average, these issues double the energy cost of the test apps. Manual verification also shows that our tests achieve a low number of false positives. Finally, we demonstrate how our test reports can facilitate developers in fixing the issues.

The key contributions of this paper are as followed:

• •

To the best of our knowledge, we conducted the largest-scale empirical study on mobile-app energy issues (largest previous study [25]: 8 app subjects, 10 energy issues; this paper: 27 app subjects, 134 energy issues). Our findings can greatly benefit the research on energy issue detection and diagnosis.

• •

Inspired by the findings, we completely implemented an automated testing framework for detecting energy issues. The key technology includes extracting battery-hungry input sequences from source code, steering the test direction on-the-fly for high detection efficacy, etc. The evaluation shows that our framework can detect a significant number of unreported energy issues (76 unreported issues from 89 apps). In contrast, present technology [2] detected only 10 unreported energy issues from 30 apps.

• •

As far as we are aware, our evaluation on a testing framework for energy issue detection is of the largest scale (largest previous study [2]: 30 app subjects; this paper: 89 app subjects). Our app subjects are also of higher quality than previous work. Our subjects are selected considering metrics such as high popularity and maintenence quality. However, most subjects in previous work can not meet this standard. Our evaluation both validates the efficacy of our framework and justifies our empirical findings.

In the remainder of this paper, we first introduce the data source for empirical study in Section 2, and discuss our findings in Section 3. Inspired by the findings, we present a testing framework for detecting energy issues. The overview of the framework will be introduced in Section 4. We then talk about the detailed technology in Section 5. Finally, we show the experimental setup and results in Section 6.

2 Data Source

Open-source projects typically have publicly accessible issue tracking systems and code repositories. In the issue tracking systems, developers can post an issue report, which contains a title and a main body part, to report the symptoms of their observed bug/issue and the steps to reproduce the issue (optional).111We may use the terms bugs and issues interchangeably in this paper. Following that, developers can discuss the issue and comment on the report. Those developers who are assigned to fix the issue can propose potential code revisions. Typically, after code review by other project members and further changes, such revisions will be committed to the project’s code repository.

Our empirical study is conducted on well-maintained Android application projects from three popular open-source software hosting platforms: GitHub222https://github.com, Mozilla333https://dxr.mozilla.org, and Chromium444https://www.chromium.org repositories. The criteria for selecting app subjects for our study are: 1) a subject should have achieved at least 1,000 downloads on the market (popularity), 2) it should have more than one hundred code revisions (maintainability). Following these criteria, we randomly selected 89 app subjects from those three software hosting platforms. These open-source applications are also indexed by the F-Droid database555https://f-droid.org/. Figure 1 gives basic information of the 89 applications. They cover most application categories in F-Droid and in total contain 63,573 issue reports, indicating that these subjects are quite large-scale.

To search for energy issues, we employed keyword searching in issue reports’ title and body to locate potential energy issues. The keywords we used are energy, power and battery. While keyword searching generally helps to retrieve most energy-related issue reports, it can also produce false positive results when the issue reports accidentally contain any of our keywords. To filter out such irrelevant issue reports, we manually verified each returned issue report to make sure the issue concerned is indeed an energy issue. In total, we checked 286 retrieved issue reports and this helped us locate 134 real energy issue reports from 27 apps. These energy issues were reported or updated from Jul 31st, 2009 to Jan 25th, 2019, and more than 70% of them were reported after Jan 1st, 2014, which guarantees the timeliness of our study.

3 Empirical Study

To answer our research questions, we carefully studied the 134 energy issue reports. The results are as follows.

3.1 RQ1: What are the common root causes of energy issues?

Among the 134 reports, 71 explicitly show the information on root causes of the issues. We examined all of them and observed the following six root causes. Some issues were caused by multiple reasons, hence, the sum of the percentages below is over 100%.

Unnecessary workload ($30/71$=$42.3\%$). Many applications perform certain computations that do not deliver perceptible benefits to users. These computations incur unnecessary workload on hardware components including CPU, GPU, GPS, data storage, network interface, and screen display. For example, in the report of Chrome issue 541612666“Chrome” is the app name, “541612” is the issue’s ID given by the corresponding issue tracking system. , the application produces frames constantly even when visually nothing is changed or repainted, which causes huge workload on CPU and GPU and makes rendering certain pages twice as energy-consuming as average. The report of Kontalk issue 304 shows that the app keeps pinging an address when device is disconnected from a WiFi network, continuously draining the battery.

Excessively frequent operations ($14/71$=$19.7\%$). Performing certain operations too frequently can also waste battery power. In comparison with unnecessary workload, when fixing energy inefficiencies caused by excessively frequent operations, the developers do not completely remove the operations (because their functionality is necessary), but reduce the frequency of the operations. For example, in Firefox (issue 979121), whenever users type in the URL bar and the text changes, the application will query the database (e.g. for auto-completion). Considering that users often visit the websites they visited before, developers suggested to store users’ web browsing history in memory to reduce database hits to save energy.

Wasted background processing ($13/71$=$18.3\%$). As battery-powered mobile devices are extremely sensitive to energy dissipation, it is good practice to make backgrounded applications as quiet as possible. Specifically, the “backgound" here means that after the use of an application or “activity" (a major type of application component that represents a single screen with a user interface777https://developer.android.com/guide/components/fundamentals.html), users press the Home button or switch to another application or activity, so the previous application or activity goes to background. A typical example is Firefox issue 1022569: when users select a new tab (each tab is an activity), the invisible old tab would still keep being reloaded by itself, which wastes battery power. Later, to fix this issue, Firefox developers proposed to remove the reloading process.

No-sleep ($18/71$=$25.4\%$). In contrast to wasted background processing issues, the no-sleep issue means that when the screen is off and device is supposed to enter sleep mode, certain apps still keep the device awake. For example, Kontalk issue 143 unnecessarily holds a wake lock [24], preventing the device from falling asleep. For another example, in Firefox (issue 1026669), the Simple Service Discovery Protocol (SSDP) is fired by the timer every two minutes when the screen is off, which not only incurs a large amount of workload but also prevents the device from entering the sleep mode. Program 1 gives the JavaScript patch for fixing this issue. It added the cases to deal with “application-background" and “application-foreground" for the SSDP service. Note that, the “application-background" defined by developers includes both screen-off time and the scenarios where users switch to another application. So this issue belongs to two categories: no-sleep and wasted background processing.

Spike workload ($2/71$=$2.8\%$). A workload spike can cause lagging UI [25], degrade user experience and heat up the device, inducing a huge energy waste. For instance, in RocketChat (issue 3321), when users send or receive .gif animation pictures, CPU utilization quickly rises to 100% and heavily affects the battery.

Runtime exception ($2/71$=$2.8\%$). In some cases, runtime exceptions may provoke abnormal behaviors of a mobile application and cause energy waste. For instance, in AntennaPod (issue 1796), the “NullPointerException" makes the download process persist and consume power. In our study, such energy issues caused by runtime exceptions are not common and we only observed two cases.

Considering that spike workload and runtime exception are of small proportions in practice, our energy-issue detection technology only focuses on the main root causes apart from them.

3.2 RQ2: How do energy issues manifest themselves in practice?

Out of the 134 reports, 63 contain explicit information that shows how the issues manifest themselves. We studied these 63 issues to answer RQ2. We observed that the majority of these issues require special inputs or a special context to trigger them, while only a few issues can be easily manifested with simple inputs.

Simple inputs ($4/63$=$6.3\%$). Simple inputs mean one tap or swipe gesture in common interaction scenarios. We found four issues are of this type of manifestation. For example, Andlytics issue 543 lets the app refresh itself whenever the user opens the app. And MaterialAudioBookPlayer issue 384 makes the app unnecessarily scan folders every time the user starts or leaves the app.

Special inputs ($46/63$=$73.0\%$). The majority of the energy issues can only be triggered with certain specific inputs or a sequence of user interactions (e.g. text typing, taps, or swipes) under certain states of an application. For instance, the c:geo (a geocaching app) issue 4704 requires three steps to reproduce: 1) open the app and make sure there is no GPS fix since the app starts, 2) change between cache details and other tabs of the same geocache, 3) put the device in standby and let timeout to screen-off. After a while, users would find GPS stays active even when the screen is turned off. To avoid energy waste, users decide to quit using the app.

Special Context ($13/63$=$20.6\%$). Special context includes environmental conditions (rather than user interactions, e.g. taps) such as the accessibility of networks, location of the device, settings of the OS and applications. In our dataset, 13 issues require such special contexts to trigger. For instance, MPDroid issue 3 appears when the user is watching stream videos but the network is disconnected; the app then keeps trying to load the video and consumes battery. AnkiDroid issue 2768 occurs when users lock the phone screen when the application is in “review" mode and a notification comes in afterwards, so the screen will hold on until the battery is dead.

4 Overview of Testing Framework

The following observations motivated us to design an automated testing framework for effectively detecting energy issues in real apps:

• •

From Finding 1, we address previously unaddressed energy issues caused by unnecessary workload and excessively frequent operations.

• •

From Finding 2, we found that 20.6% of energy issues can only be manifested under special context such as poor network performance. However, such factors were previously neglected.

According to the first observation, as we discussed in Section 1, evaluating the necessity of app workload is crucial for identifying these issues. We will use machine learning to help us cluster the workloads, and further assess its necessity, as shown later in Section 5.3. According to the second observation, we will devise two types of most common special contexts for effectively revealing these issues, as shown in Section 5.1.2. Developers also can duplicate our experiment for detecting these hidden issues. Importantly, we also make practical designs and implementations to enhance the efficacy of our testing framework.

Figure 2 shows the framework overview. The framework first makes sophisticated preparation before testing. For example, it inspects the source code and collects the candidate input-sequences that are most suspected of energy overuse. A set of candidate runtime contexts (containing the above mentioned two types of special contexts) are also carefully designed to increase the chance of provoking energy issues. Later, these candidate inputs and contexts will be explored under an effective and systematic scheme.

To start a test case, the framework at first clears unrelated threads and data to minimize the interference from other applications and previous test cases. Afterwards, it will select one input sequence and one runtime context from the candidates, then run the app with them. During the entire test, the power consumption of device is traced with a power monitor. Our framework will look into the power trace and decide whether an energy issue exists. If one does exist, the issue information will be added into database. The entire test is limited with a time budget. If the time budget runs out, the test will quit and test reports will be generated for developers to help fix the issues. Otherwise, our framework will start a new test case.

As mentioned above, our framework explores the inputs and contexts under a systematic scheme, where the exploring direction is tuned on-the-fly. The rationale of our scheme is that if an energy issue occurs, it implies that the type of the input sequence and runtime context incurring this issue may be more likely to uncover energy issues than average case, since it did cause an energy issue to show up; we thus increase the chance of this type of inputs and context to be tested. Concretely, we utilize a set of parameters and iteratively update them to guide the test direction, as shown later in Section 5.2.

The large-scale evaluation (Section 6) shows that, exploiting these practical and targeted tests, our framework largely outperforms the state of the art on the efficacy in detecting all kinds of energy issues.

5 Detailed technology

This section will introduce the detailed technology of implementing the testing framework. It involves how to design candidate input sequences and running contexts, how to steer the test direction at runtime, and how to identify energy issues from the power traces, etc. The ultimate objective of our technology is to effectively and accurately pinpoint energy issues.

5.1 Preparation before testing

Our candidate input sequences are designed with high utilization of main hardware components, such as CPU, screen display and network interface, since they are usually the culprits of energy over-use, as shown in the literature [30]. At the same time, we will devise a set of artificial running contexts which are most likely to trigger energy issues.

5.1.1 Design of candidate input sequences

We design two types of candidate input sequences. One is weighted input sequences, the other is random input sequences.

Weighted input sequences are generated referring to the Event-Flow Graph (EFG) [27]. Each node in EFG is a User Interface (UI) component, such as a button or a list item. If a user interaction on a UI component, say $node_{1}$, can immediately activate another UI component, say $node_{2}$, then EFG should have a directed edge from $node_{1}$ to $node_{2}$. Technically, we utilize Layout Inspector888https://developer.android.com/studio/debug/layout-inspector to construct the EFG of an app. So an arbitrary path in EFG is a candidate input sequence for our testing framework. In practice, our test cases always start from the root node (i.e., the beginning UI component of the app). And the lengths of paths are constrained with a limit. Note that, Layout Inspector can construct EFG, but can not run apps with the paths. So later in testing, we use Dynodroid999https://dynodroid.github.io to feed the paths to apps.

More importantly, all the candidates generated from EFG are assigned with a weight. The weight indicates the potential of a sequence to cause energy waste. And the input sequence with a larger weight has a higher priority to be tested. We adopt Equation (1) to calculate the weight for each input sequence. $S$ is the number of system APIs (Application Program Interfaces) invoked by the input sequence. $C$ is the number of function invocations and block transitions incurred by the input sequence. $\alpha$ and $\beta$ are used for adjusting the influence of $S$ and $F$; $\alpha>0,\beta>0,\alpha+\beta=1$.

[TABLE]

The reason why we resort to $S$ and $C$ to indicate the potential of causing excessive energy use is the following: as shown in [31], main energy-consuming components in smartphones are CPU, screen display, network interface (cellular and WiFi), further, GPS and various sensors. Except for CPU, all other components can only be controlled by system APIs. More system APIs an input sequence accesses, larger chance it may cause energy waste. On the other hand, CPU is dedicated to executing basic operations that constitute the source code of apps, for example, arithmetic operations like additions and multiplications, and control-flow operations mainly including function invocations and block transitions. The literature [21, 18] has shown that control-flow operations are the main energy-consumers for Java source code, which take up more than one third of the CPU energy cost. We therefore use the total number of function invocations and block transitions to indicate the potential CPU overuse of an input sequence.

Note again that, we calculate $S$ and $F$ before testing. We first instrument the app source code, and run the app with the input sequences, and then count out their $S$ and $F$ individually.

Apart from weighted input sequences, we also designed random input sequences to cover some random cases we might not envisage. We use the Monkey tool (an Android UI/application exerciser101010https://developer.android.com/studio/test/monkey.html.) to automatically generate random input sequences, such as taps and swipes. “Random" means the position of the inputs on screen are randomly set. Monkey does not generate input sequences at runtime. Instead, Monkey pre-defined a large number of random input sequences. When being asked for an input sequence, Monkey will randomly deliver one of them with its ID (called seed in Monkey). The advantage of this design is that testers can always use the same ID to repeat the input sequence and reproduce the test case. In our test, the pre-defined input sequences in Monkey are all adopted as our candidate input sequences.

In summary, our test combines the two types of input sequences, namely, weighted and random. The former will help us detect energy issues in an effective means, and the latter will cover corner cases beyond our considerations. Later in Section 5.2, we will show the strategy on how to balance these two types of input sequences.

5.1.2 Design of candidate running contexts

The entire experiment is set in a signal shielding room. We are enabled to manipulate the contextual factors, such as the strength of network and GPS signal. In our experiment, we designed three types of running contexts, namely, Normal, Network Fail and Flight Mode. In Normal, the network and GPS both work normally (package delivery delay is 36 ms and download bandwidth is 3.2 Mb/s). In Network Fail, the signal of network and GPS is seriously weak (package delivery delay lengthens to 451 ms, download bandwidth drops to 12.0 Kb/s). In Flight Mode, the network is closed at software level by operating system, but GPS works normally.

The reason why we choose Network Fail and Flight Mode as representatives for special contexts is that, our empirical study shows they are the two major types of special contexts. The former holds 25.0% (3 out of 12), the latter occupies 16.7% (2 out of 12) of all issues manifested under special context.

We also designed a special type of running context, Non-background. We designed this context because in our experiment we observed that it can provoke more no-sleep issues, as shown later in Section 6.4. In Non-background, the network and GPS work ordinarily, however, we do not input a press of Home button to the device after EXECUTION stage (the stage-division for test cases will be explained in Section 5.3). That is, the test case does not have BACKGROUND stage, and straight goes to SCREEN-OFF.

5.2 Steer the test direction on-the-fly

Our framework steers the test direction dynamically based on test history. Algorithm 1 shows details of our steering scheme. The rationale behind is this: when an energy issue is detected, it implies that this type of input sequence and running context may have a larger opportunity to provoke energy issues than normal case since it did bring an energy issue to light. Hence, our framework will generate slightly more of this type of test cases for larger chance of detecting energy issues.

Data for the algorithm

The candidate input sequences and running contexts are designed based on the approach we demonstrated in Section 5.1.1 and 5.1.2. We present them in the data structures of $WeightedInputSequences$, $Random$ $InputSequences$ and $RunningContexts$. $N$ is the number of candidate running contexts. In our experiment, we devised 4 running contexts, so $N==4$.

$p_{wei}$ and $P_{ctx}$ are the steering parameters for directing the test. $p_{wei}$ is the probability of choosing a weighted input sequence for the upcoming test case. In our experiment, we initialize it as 50%, so the test case has a chance of 50% to run with weighted input sequence at the beginning of test, and we will update and refine $p_{wei}$ on-the-fly during the entire testing. On the other hand, one element $p_{k}$ in $P_{ctx}$ represents the probability of choosing $context_{k}$ in $RunningContexts$. We will also update $P_{ctx}$ at runtime. Note that, the summation of the elements in $P_{ctx}$ is bound to 1.

$\Delta_{wei}$ is the increment utilized to increase or decrease $p_{wei}$ to refine $p_{wei}$. $\Delta_{context}$ plays the same role for $P_{ctx}$. Larger $\Delta_{wei}$ and $\Delta_{contex}$ we employ, more aggressively we tune the test direction.

Details of the algorithm

We first prepare the data and initialize the parameters (i.e., $p_{wei}$, $P_{ctx}$, $\Delta_{wei}$ and $\Delta_{context}$). We then start a test case, clear unrelated threads and data, launch the app from root screen. Next, we will decide the type of input sequence; weighted input sequences have a probability of $p_{wei}$ to be chosen. If the chosen type is “weighted", we then select an unexplored sequence with the highest $weight$ in $WeightedInputSequences$. Otherwise, we randomly select one from $RandomInputSequences$. Likewise for running context, we select one from RunningContexts with its corresponding probability.

We feed the app subject with the selected input sequence and running context on device, and measure the power consumption at the same time. The power trace will be analysed to confirm whether an energy issue occurs. If there exists an energy issue, it implies that the corresponding type of input sequence and running context may be profitable for provoking more energy issues, our testing framework then steers a bit to this direction. Specifically, if it is triggered with a weighted input sequence, we increase $p_{wei}$ by $\Delta_{wei}$, and $p_{wei}$ should not exceed the $wei\_up$ $\_threshold$. Otherwise, we decrease $p_{wei}$ by $\Delta_{wei}$. Also, we keep $p_{wei}>=wei\_down$ $\_threshold$.

An analogous approach is applied to refining $P_{ctx}$. We check under which running context (e.g., $context_{k}$) the issue occurs, then increase its testing probability (e.g., $p_{k}$). However, the precondition is that there should be at least one elements (except $p_{k}$ itself) in $P_{ctx}$ that are above $cxt\_down$ $\_threshold$ because we, on one hand, intend to rebalance the probabilities, on the other hand, we should also let all contexts have at least a possibility of $cxt\_down\_threshold$ to be tested.

5.3 Identify energy issues from power trace

We divide the power trace into five stages, namely, PRE-OFF, IDLE, EXECUTION, BACKGROUND and SCREEN-OFF. This division can help us identify three types of energy issues: execution issues (including issues caused by unnecessary workload and excessively frequent operations), background issues and no-sleep issues. Figure 3 shows an illustration of power traces with these three types of energy issues.

PRE-OFF stage is the beginning stage where the device is powered but the screen is off. Then, the test case will be transferred to IDLE stage by turning on the screen. To enter EXECUTION stage, the subject application will be opened and run with a certain input sequence and running context, which are generated as shown in Section 5.1 and 5.2. After EXECUTION stage, the application will be fed with a press of Home button to enter BACKGROUND stage. The final stage is SCREEN-OFF stage, which begins when the screen is supposed to be turned off automatically, however, a part of energy issues will keep the screen on even at SCREEN-OFF stage, eating a large amount of battery power.

Identifying execution issues

For execution issues, as we discussed in Section 1, evaluating the necessity of app workload is crucial for identifying them. Specifically, we employ the Dbscan clustering algorithm [7] (Density based spatial clustering of applications with noise) to fulfil this purpose. The objective of Dbscan is to classify multidimensional data points into three groups, namely, core points, border points and outlier points. After clustering, the data points should have the following properties:

1. For a core point, the number of its neighbours (the points within a range of $\varepsilon$ from it) is no less than a certain value, $MinPts$. Generally speaking, the core points are “quite close and gathered".

2. For a border point, its neighbours are less than $MinPts$, but it is a neighbour of at least one core point or another border point.

3. For an outlier point, its neighbours are less than $MinPts$, and it does not have either a core or a border neighbour.

We treat each test case as a data point, and treat the test cases in the same app category as a data set for clustering. The dimensions of each data point we employed for clustering are $l_{chpp}$, $n_{chpp}$, $\mu_{chpp}$, $\mu_{exe}$, which are all extracted from power trace of EXECUTION stage of each test case. $l_{chpp}$ is the total length of continuous-high-power periods. Continuous-high-power period is when power continuously exceeds a certain threshold longer than a certain length. $n_{chpp}$ is the number of these periods. $\mu_{chpp}$ is the average power of these periods. $\mu_{exe}$ is the average power of the entire EXECUTION stage.

Dbscan then classifies the test cases into those three groups. We label the test cases in core and border groups as “normal", and label the ones in outlier group as suspects for suffering from energy issues. Later, our evaluation on 89 apps (involving 35600 test cases) shows that only 1.1% test cases are outliers, which are observably energy-consuming. We assume they have “unnecessary" workloads or operations. Finally, we manually verify whether there exist real execution issues. In the experiment, our testing framework detected 47 candidate execution issues from those 1.1% test cases. Only three (out of the 47) are false positives, indicating the high reliability of this approach. In contrast, current technology [2] detected 3 candidate execution issues from 30 apps, and still one of them is a false positive.

Identifying background and no-sleep issues

If the app is free from background issues, the power trace in BACKGROUND stage is supposed to be similar to that in IDLE stage. We thus compute the dissimilarity value of the two traces. If the value is above a certain threshold (40% in our experiment), we label this test case as a candidate for a background issue.

We identify the no-sleep issues in the same way. We compare PRE-OFF with SCREEN-OFF. If the dissimilarity exceeds a certain threshold (50% in our experiment), we speculate this test case is suffering from a candidate no-sleep issue.

5.4 Manual verification

After the candidate issues are found, we manually verify whether they are actual energy issues. The steps for manual verification are as followed:

a. We re-run the test case to check if the energy issue can be reproduced.

b. We analyse the power trace to affirm its significant impact on energy consumption.

c. We observe symptoms of the issue, including flickering UI, popping-up messages, long CPU wake-time, high CPU utilization and high GPU rendering speed.

d. We inspect the execution trace (obtained from Android Traceview111111https://developer.android.com/studio/profile/traceview, now Traceview is deprecated, developers can use CPU Profiler121212https://developer.android.com/studio/profile/cpu-profiler instead) to check if there has frequently-provoked methods in the program. In practice, they are usually the suspects of defective code. According to the issue reports in our empirical study, for the issues diagnosed using this information, the faulty code was 100% found and fixed. Here are two examples131313https://bugs.chromium.org/p/chromium/issues/detail?id=349059,141414https://bugs.chromium.org/p/chromium/issues/detail?id=480522.

Only if the issue can be reproduced, cause noticeable energy waste, have visible symptoms, and its faulty code can be located, we affirm its real existence. Otherwise, we determine it as a false positive.

6 Experimental Evaluation

In this section, we first present the specifics of our experimental setup. Then, we evaluate our testing framework on various aspects, such as its efficacy of detecting energy issues, its comparison with the state-of-the-art, etc. The result shows that our testing framework largely outperforms the present technology, which benefits from the sound empirical study and targetting implementations of the testing framework.

6.1 Experimental setup

We employ the Odroid-XU4 development board151515https://wiki.odroid.com/odroid-xu4/odroid-xu4, whose processor has four big cores with a frequency of 2 GHz and four small cores with a frequency of 1.3 GHz. The main memory is as large as 2 GB. The board possesses a powerful 3D accelerator, Mali-T628 MP6 GPU. The high capacity of Odroid-XU4 board guarantees its performance for most applications on the market.

It also provides a package of developer options. For example, it can display CPU utilization and GPU rendering profile on screen which will aid us in verifying the real existence of energy issues. It is also equipped with a power monitor, Smartpower2161616https://wiki.odroid.com/accessory/power_supply_battery/smartpower2, to measure the real-time power consumption. The sampling rate is 100 Hz. Due to these rich features, Odroid board is widely-employed in the field of energy optimization for mobile devices [32][40].

We use Android as our target operating system since it is open-sourced and its market share captures around 85.9%171717https://www.statista.com/statistics/266136/global-market-share-held-by-smartphone-operating-systems/ of the worldwide smartphone volume by the end of first quarter of 2018. We evaluate our framework on the same set of 89 app subjects in the empirical study. Our total testing time for the 89 app subjects is 2373.3 hours, i.e., 98.9 days (on average, 1.11 days for one app subject).

6.2 The efficacy of our testing framework

The experimental result shows that our test detected 91 candidate energy issues, among which we manually confirmed 83 real energy issues. We have 8 false positives. The false positives are caused by the noise in power samples in some case. The noise could be a result of OS activities at background, such as Android Runtime Garbage Collection181818https://source.android.com/devices/tech/dalvik/gc-debug, random system downloading and updating.

Table 2 shows 9 examples of the detected energy issues. For instance, in Leisure, three animated .gif are loaded and playing at the bottom of one certain page even though they are invisible to users most of the time. This execution issue wastes 25.9% energy use. It can be fixed by freezing the animation when the .gif pictures are not shown on the screen. For another example, when Chess Clock is after use and backgrounded, the device will be heated up from $41.2^{\circ}$C to $60.9^{\circ}$C due to the inefficient and long utilization of CPU. The average power of this issue is 59.2% higher than that of the IDLE stage.

91.6% (76 out of 83) detected energy issues in our test are newly-reported. Our experiment also shows that these issues averagely double the energy consumption of the apps (see later in Section 6.5). Without our test, these serious energy issues would have never surfaced even though the battery drains desperately.

On the other hand, 94.5% (120 out of 127) energy issues in our empirical study were not listed in the issues detected by our test. Considering that the test is very large (2373.3 hours), it is infeasible to go through all the test cases to find the reasons why they were not detected, and present their proportions. However, we conjecture the major reasons may be the following:

Firstly, our standard of determining an energy issue is much higher than that of developers. The issues detected by our test usually have long-lasting impact on the energy consumption, the energy waste is usually above 10.0%. However, many issues detected by developers may only cause a transient workload, and their energy waste can be hardly more than 10.0%. Secondly, a number (26.7%, 32 out of 120) of the issues are not reproducible, so our test can not trigger them either. Thirdly, due to the time limit, the variety of input sequences and running contexts in our test is not large enough to cover all of them.

After all, our framework still shows its efficacy in detecting severe and unreported energy issues, which were beyond the vision of researchers formerly.

6.3 Comparison with the state-of-the-art

We will compare the efficacy of our testing framework with the present technology [2]. Since most of their app subjects ($18/30=60.0\%$) are not open-sourced, we cannot look into the execution trace to verify the issues. Also, similar to the comparison with developers, our standard of identifying energy issues is much higher than that of the former researchers, we then do not make comparison on the set of their subjects. Instead, we make it statistically.

We use the efficacy ratio, i.e. the ratio ($r^{d}_{s}$) of the number of detected issues to the number of subject apps, to indicate the efficacy of a testing framework. As shown in Figure 4, for execution issues, their $r^{d}_{s}$ is $6.7\%$ (2 issue out of 30 apps), our $r^{d}_{s}$ is $49.4\%$ (44 out of 89), for background (BG) and no-sleep (NS) issues together, their $r^{d}_{s}$ is $33.3\%$ (10 out of 30), ours is $43.8\%$ (39 out of 89). We put background and no-sleep issues together since their work did not distinguish them. The comparison validates that our testing framework is more effective than the state-of-the-art technology. It is because our work is based on the insightful empirical findings, rather than an ungrounded assumption.

6.4 Issue cause and manifestation

Figure 5 demonstrates breakdown of energy issues of different causes in empirical study and experiment. “UW" is unnecessary workload, “EFO" is excessively frequent operations, “BG" is wasted background processing, “NS" is no-sleep, “SW" is spike workload, “RE" is runtime exception. It shows that UW and EFO are very significant root causes of energy issues in both the empirical study and the experiment, which justifies our Finding 1.

Figure 6 presents the numbers of energy issues provoked under different context, which captures manifestation characteristics of energy issues. We can see that most execution issues are manifested under the Normal context because many scenarios where issues occur require normal network and GPS context to reach. For example, as we talked above, the issue in Leisure occurs only when those three .gif pictures were downloaded and showing on the page.

We only have four ($4/83=4.8\%$) background issues, which indicates that operating system is competent in clearing the influence of backgrounded apps at BACKGROUND stage. However, backgrounded apps may still suffer from no-sleep issues: Normal, Network Fail and Flight Mode provoke 6, 17, 13 no-sleep issues respectively. Of particular note is that even though Non-background and Normal are running under same context, the former inclines to provoke more no-sleep issues. In our test, it brings on 17 issues. This result implies that special contexts (and Non-background) tend to incur anomalous behaviours of apps, such as bad use of wake lock, and thus cause more no-sleep issues.

Figure 6 also presents that 37.3% (31 out of 83) energy issues can only be triggered under Network Fail and Flight Mode. This confirms the Finding 2: special context, such as network fail, hides a significant number of serious energy issues.

6.5 Energy waste of the detected issues

Figure 7 shows the energy waste of detected energy issues. The energy waste is calculated using Equation (2).

[TABLE]

$w$ is the energy waste of the issue, $e_{x}$ is average power of the corresponding stage in the test case with the corresponding issue (EXECUTION stage for execution issues, BACKGROUND stage for background issues, SCREEN-OFF stage for no-sleep issues). $e_{n}$ is “normal" energy cost. We define “normal" energy cost individually for different issues. For background and no-sleep issues, we adopt average power at IDLE and PRE-OFF stage as normal cost, respectively. For execution issues, we use mean value of average powers of EXECUTION stage in test cases in the same app category, as normal cost.

The experimental result shows that, the energy waste of execution (Exe) issues is 25.0% on average and up to 137.6%, the energy waste of background (BG) issues is 88.9% on average and at maximum 196.2%, the values for no-sleep (NS) issues are 200.0% and 663.3%, separately. Overall, the average energy waste of the issues is 101.7%.

6.6 How can our framework benefit developers?

After identifying the energy issue, our framework generates a test report to help fix the issue. The report includes:

• •

The information on input sequence and running context, and screen-casting video recording the test case. Having them, developers can analyse the manifestation and symptoms of the issue.

• •

The visualized power trace of the test case. With it, developers will have an intuitive view on power consumption of the issue.

• •

The method-level execution trace provided by Android Trace View. The issue reports in empirical study showed that this information can always ($100\%$) help find the faulty code.

• •

The rationale behind our testing framework. Developers will understand why and how the testing framework works.

We also conducted a case study, showing how to utilize the test report to diagnose and fix energy issues. We take the execution issue in Leisure and the no-sleep issue in Vanilla (as we listed in Table 2) as two examples. We reproduce the issues, observe the symptoms, check the power traces (the red dash lines in Figure 8 and 9). We then find the frequently-called methods in the source code that result in energy waste, and refactor the code to remove the unnecessary workload and release the wake-lock after use. As we can see, the power traces (black full lines in Figure 8 and 9) after fixing are much flatter or lower than those before fixing.

7 Related Work

Our work is related to multiple lines of research work. We will discuss in three areas: understanding energy issues; detecting & diagnosing energy issues; fixing energy issues & optimizing software.

Understanding Energy Issues

The first empirical study on characteristics of energy issues in the system of mobile device was done by Pathak et al. [30]. They mined over 39,000 posts from four online mobile user forums and mobile OS bug repositories, and studied the categorization and manifestation of energy issues. These studied issues involve multiple layers across the system stack of mobile devices, from hardware, operating systems to applications. In comparison, our empirical study in this paper focuses on app-level energy issues.

The above style of empirical study that mines the data in project repositories has been widely applied. For example, to characterize performance issues, a large body of research has been done for PC and server side software [13, 29, 43, 25]. The most related study is conducted by Liu et al. [25] that investigates performance issues (including energy issues) in Android applications. However, the number of the studied energy issues is very small compared with our study. More importantly, our study is only targeted at energy issues, so our findings are more specialized and comprehensive on energy issues.

Detecting & Diagnosing Energy Issues

To detect abnormal energy use of mobile apps, researchers use operating system and hardware features as predictors to infer energy information at device, component, virtual machine or application level [15, 14, 31, 17, 34, 42, 28].

Shuai et al. [9] and Ding et al. [19] proposed approaches to obtaining energy information at source line level. The former requires the specific energy profile of the target systems, which may not be accessible to developers. The latter utilizes advanced measurement techniques to obtain source line energy cost.

The work [2] is the most relevant to ours. Even though their technology is competent in detecting background and no-sleep issues, two key differences prevent it from uncovering most serious energy issues detected in our test. The first difference is that their work could not deal with execution issues properly, as we discussed in Section 1. The second difference is that their work did not take the special running context into consideration. As shown in our experiment, 37.3% (31 out of 83) energy issues can only manifest themselves under special context. In summary, owing to our high-quality empirical study and well-designed testing technology, our framework surpasses the state-of-the-art in detecting all kinds of energy issues.

Fixing Energy Issues & Optimizing Software

A large amount of research effort on energy-saving for mobile devices has been focused on the main hardware components, such as the CPU, display and network interface. The CPU-related techniques involve dynamic voltage and frequency scaling [6], heterogeneous architecture [22, 8, 20] and computation offloading [16, 37]. Techniques targeting the display include dynamic frame rate tuning [11], dynamic resolution tuning [10] and tone-mapping based back-light scaling [12, 1]. Network-related techniques try to exploit idle and deep sleep opportunities [23, 36], shape the traffic patterns [5, 33], trade-off energy against other criteria [41, 35, 4] and so on. Such work attempts to reduce energy dissipation by optimizing the hardware usage; on the other hand, several pieces of work aim at designing new hardware and devices [38, 39].

Two pieces of work [3, 26] provide systematic approaches to optimizing software source code. In the former, Boddy et al. attempted to decrease the energy consumption of software by handling code as if it were genetic material so as to evolve to be more energy-efficient. In the latter, Irene et al. proposed a framework to optimize Java applications by iteratively searching for more energy-saving implementations in the design space.

8 Conclusion

In this paper, we conducted an empirical study on software energy issues in 27 well-maintained open-source mobile apps. Our study revealed root causes and manifestation of energy issues. Inspired by this study, we fully implemented a novel testing framework for detecting energy issues. It first statically analyses the source code of app subjects and then extracts the candidate input-sequences with large probability of causing energy issues. We also devised several artificial runtime contexts that can expose deeply-hidden energy issues. Our framework effectively examines apps with the inputs and contexts under a systematic scheme, and then automatically identifies energy issues from power traces. A large-scale experimental evaluation showed that our framework is capable of detecting a large number of energy issues, most of which existing techniques cannot handle. These issues on average double the energy cost of the apps. Finally, we showed how developers can utilize our test reports to fix the issues.