Non-Functional Testing of Runtime Enforcers in Android

TL;DR

This paper presents a practical method combining automatic test generation and runtime monitoring to identify performance issues in Android runtime enforcers, focusing on responsiveness, launch time, memory, and energy consumption.

Contribution

It introduces a novel approach for non-functional testing of runtime enforcers in Android, addressing the challenge of verifying performance characteristics.

Findings

Detects performance issues in automatically generated enforcers

Effective in monitoring responsiveness, launch time, memory, energy

Identifies inefficiencies introduced by runtime enforcers

Abstract

Runtime enforcers can be used to ensure that running applications satisfy desired correctness properties. Although runtime enforcers that are correct-by-construction with respect to abstract behavioral models are relatively easy to specify, the concrete software enforcers generated from these specifications may easily introduce issues in the target application. Indeed developers can generate test suites to verify the functional behavior of the enforcers, for instance exploiting the same models used to specify them. However, it remains challenging and tedious to verify the behavior of enforcers in terms of non-functional performance characteristics. This paper describes a practical approach to reveal runtime enforcers that may introduce inefficiencies in the target application. The approach relies on a combination of automatic test generation and runtime monitoring of multiple key performance indicators. We designed our approach to reveal issues in four indicators for mobile systems: responsiveness, launch time, memory, and energy consumption. Experimental results show that our approach can detect performance issues that might be introduced by automatically generated enforcers.