In Situ Network and Application Performance Measurement on Android Devices and the Imperfections

TL;DR

This paper investigates how energy-aware hardware tuning and VPN-based measurement tools impact network and application performance on Android devices, revealing that these factors can distort performance assessments and emphasizing the need for comprehensive understanding.

Contribution

It provides an analysis of how system optimizations and measurement tools influence performance metrics, highlighting challenges in accurate in situ performance measurement on Android.

Findings

Energy-aware hardware tuning affects application performance and network throughput.

VPN-based measurement tools can distort network performance assessments.

Measurement accuracy depends on understanding device, network, tools, and applications.

Abstract

Understanding network and application performance are essential for debugging, improving user experience, and performance comparison. Meanwhile, modern mobile systems are optimized for energy-efficient computation and communications that may limit the performance of network and applications. In recent years, several tools have emerged that analyze network performance of mobile applications in~situ with the help of the VPN service. There is a limited understanding of how these measurement tools and system optimizations affect the network and application performance. In this study, we first demonstrate that mobile systems employ energy-aware system hardware tuning, which affects application performance and network throughput. We next show that the VPN-based application performance measurement tools, such as Lumen, PrivacyGuard, and Video Optimizer, aid in ambiguous network performance measurements and degrade the application performance. Our findings suggest that sound application and network performance measurement on Android devices requires a good understanding of the device, networks, measurement tools, and applications.